Welcome to Measuring the Immeasurable, a newsletter helping arts and culture leaders understand the language of public value. Here, I provide what I hope is a useful distinction between evaluation-as-measurement and evaluation-as-assessment. This isn’t an academic distinction and has real consequences for how our work is valued. If you find this work useful, please consider sharing and subscribing.
I’m on the Board of the Social Impact Measurement Network Australia, or SIMNA. We’re a professional network of people who cluster around the elusive idea of ‘public value’ and related concepts like ‘evaluation,’ ‘outcomes assessment,’ and ‘social impact.’ We’re concerned with improving public policy through evidence-informed decision making.
An issue I’ve raised on the Board concerns the use of the term ‘measurement’ in our name. As an artist and arts evaluator, the idea of measuring art makes me deeply uncomfortable. In the 14th Century, the root word for measure, ‘mesuren,’ was used in the context of things which could be measured quantitatively. We can measure a child’s growth by marking their height on the wall with a pencil. We can measure the weight of two sacks of grain by putting each on a scale. We can measure the financial value of a business decision by comparing two quarterly reports.
Measurement means quantification, and quantification means comparison. Measurement strives to be objective, to be able to tell us conclusively that one thing is greater-than or less-than another. In the arts, we often experience this pressure to explain our work in quantitative terms, and we rightly experience it as an absurd demand. How on earth can we compare an artist’s residency for a jazz musician to a culture-first Indigenous wellbeing program? Is a production of Cosi half as good if half the theatre is empty1?
At SIMNA, we have this wavy little ribbon in our annual awards logo. It is meant to represent a measuring ribbon, like what a fashion designer uses to take someone’s measurements. Although evaluation and impact assessment have come a long way over the years — developing methods and tools that make use of qualitative data to tell complex stories — the idea of quantitative measurement remains central to our identity, like thick rope keeping us tied to a pier.
In my own work, I use the term assessment. The term assessment derives from the Latin word assessus, which means ‘to sit beside.’ Assessment, unlike measurement, doesn’t aim at providing an objective, comparative statement about the value of something. The term ‘assessment’ acknowledges that social impact and evaluation can only ‘sit beside’ good governance and decision-making. As I wrote a few weeks ago, social impact assessment is “a form of evaluation which knows its limits2.”
At SIMNA, I find myself in the belly of the beast having these conversations. The logic of measurement still has a significant grip on our public imagination. One of our roles as artists is to point out the limits of what we can measure. As the philosopher Herbert Marcuse argued3:
“The truth of art lies in its power to break the monopoly
of established reality to define what is real.”
Today, measurement often defines what is real. When artists shift from measurement to assessment, we can find ways of talking about art which can sit beside it in all of its beauty and complexity.
According to the DSLGC in 2014, the answer to this question is ‘yes.’ The PVMF was a harbinger for much of arts measurement culture today.
See One Dimensional Man.
Great post! I share your concerns and sometimes write about similar issues. Great to have a fellow traveller here on Substack!