Know What You Care About
Evaluation as Allocating Care
Welcome to Measuring the Immeasurable, a newsletter helping cultural leaders understand the culture of evaluation. Before jumping in to this piece, I want to share I’ve set myself a ‘100 coffees’ challenge by the end of the year. I want to hear from arts and cultural workers how digital, data, evaluation, grantwriting and accountability are showing up in your world.
If you’re in the arts and culture sector and would like to catch up for coffee (real or virtual), please message me on LinkedIn, send me an email (you can just reply to this post), or book in a phone call here (classic phone call - I’m sick of Zoom!).
Today’s post explores evaluation as a method for determining what we care about. If you also find it useful, please consider sharing and subscribing.
Embedding evaluation in your work doesn’t need to be complicated, but that isn’t to say it is easy. The good-enough framework, particularly for a small-to-medium arts organisation, might fit on a single sheet in an Excel workbook. However, determining what is worthy of being included on that page is challenging. The Shining by Stanley Kubric had over 1 million feet of film on the cutting room floor.
In my most recent newsletter, I shared the idea that a data framework can be thought of as a net. The holes in a net aren’t a weakness, but a constitutive feature. The gaps in a framework are there because we have given consideration to what we are willing to leave behind, exclude, omit, and disregard in holding ourselves (or each other) accountable. The things we choose not to measure aren’t necessarily unimportant, they’re just not important enough to be worth our time and energy. A data framework can be a way of an organisation picking its battles.
I’m generally wary of ‘life hacks,’ but there is one I have found particularly helpful: When a friend or colleague says they “haven’t got the time,” for something, I rephrase it in my mind as: “I am unwilling to prioritise this.” This isn’t a value judgment on the friend or colleague, but just a recognition that they only have twenty-four hours in a day. Every artist and arts organisation also works under conditions of finite resourcing1. So, the life hack works here in reverse: You can tell what an organisation values by what it chooses to measure.
Evaluation, then, can be a way of re-encountering that our resources (time, energy, and people) are finite. It is an occasion to critically consider and choose what we want to focus on. Unlike a vision or mission, which tends to rot in our Strategic Plans, evaluation forces us to make our priorities tangible. When I say evaluation isn’t easy, this really is the tricky part. Arts organisations value many things — the wellbeing of their artists, their audience’s experiences, the development of the broader sector, the extended social impacts that result from their work, and, in a few honourable cases, an increasing number are particularly conscious of their impact on the planet. For a small organisation, measuring all these things can be burdensome. Evaluation helps us identify what is most worthy of our attention.
An example here may be helpful. I developed a framework for the West Australian Youth Jazz Orchestra (WAYJO) several months ago. WAYJO is a small organisation of about 2-3 full-time equivalent staff. In the eyes of their funders, they are a producing and presenting organisation, which means their primary job is to put on work for audiences. However, there was a key moment in our discussions where I asked if they had ever encouraged a long-term, accomplished and highly skilled artist to leave the program to pursue the next phase of their career. They reflected that indeed, that did happen from time to time. This, of course, would be anathema to any organisation focused on artistic excellence and audience experience: it would be shooting yourself in the foot.
From our conversations it emerged that WAYJO’s focus really was on artist development. More specifically, it was on their long-term career development (musical and creative development mattered, but comparatively less). This gave our data framework a much tighter focus on measures like diversity, equity, retention, and long-term career success. It also made the work manageable and highly relevant to WAYJO’s priorities.
The good-enough framework is a lean, simple statement about what an organisation values. At a minimum, it is a wonderful communication tool to credibly argue for the value of the work you do. However, I find my work most satisfying when it becomes a way for an organisation to see itself by seeing what it cares about.
It’s worth noting, also, that this finitude cannot be resolved by more funding. No amount of funding ever removes the fact that funding is finite. Having ‘enough’ is more of a mindset than a quantity, whether we’re talking about funding, data, or anything else.