Understanding Impact Strategy
How Evaluation Supports and Sustains Strategic Planning
Welcome to Measuring the Immeasurable, a newsletter helping cultural leaders understand the culture of evaluation. Before jumping in to this piece, I want to share I’ve set myself a ‘100 coffees’ challenge by the end of the year. Thanks to those of you who have reached out already. I want to hear from arts and cultural workers how digital, data, evaluation, grantwriting and accountability are showing up in your world.
If you’re in the arts and culture sector and would like to catch up for coffee (real or virtual), please message me on LinkedIn, send me an email (you can just reply to this post), or book in a phone call here (classic phone call - I’m sick of Zoom!).
Today’s post explores the elusive idea of ‘Impact Strategy’ and invites organisations to step up in designing, and owning, their own evaluation capacity. If you also find it useful, please consider sharing and subscribing.
In my last post I spoke about how organisation-led evaluation can ‘force [arts organisations] to make [their] priorities tangible.’ Evaluation asks us to make nuanced decisions about where to best place our efforts:
“Arts organisations value many things — the wellbeing of their artists, their audience’s experiences, the development of the broader sector, the extended social impacts that result from their work, and, in a few honourable cases, an increasing number are particularly conscious of their impact on the planet. For a small organisation, measuring all these things can be burdensome. Evaluation helps us identify what is most worthy of our attention.”
You’ll forgive me for quoting myself here. A point I’d like to highlight here is this kind of challenge only really emerges when an organisation takes responsibility for evaluation. Consultant-led reporting to meet stakeholder demands rarely requires any serious organisational introspection. When a time- and resource-limited organisation begins to take responsibility for evaluation, however, it is inevitably required to make difficult decisions about what it values. This process of deciding what we value enough to measure and assess is often as valuable as the data itself.
When an organisation begins to take evaluation seriously, it starts to look and feel a lot like strategy. This is why more and more cultural organisations are producing ‘Impact Reports’ rather than ‘Annual Reports.’ A one-page evaluation framework looks a little bit like a ‘Strategic Plan on a Page,’ perhaps viewed from a different angle or another lens.
However, there are key differences. Where a strategic plan is forward-looking and aspirational, the evaluation framework is a little cooler and more discerning. The two have a sort of head & heart relationship, like twin protagonists of a feel-good comedy. The strategic plan lays out what we want to do, but the evaluation framework will quietly hold us to account on our aspirations.
I think it’s fair to call the strategic plan the ‘heart,’ here in as much as it can claim bold, long-term intentions but it is rarely held accountable to them. The strategic plan is a Pollyanna: It claims more stability, certainty, and linear progress than is reasonable. As Roger Martin, former dean of the Rotman School of Management at the University of Toronto put it: A plan is not a strategy. A strategy is for when things don’t go according to plan. A plan is good for executing a series of steps to accomplish an outcome in a relatively stable environment, but that does not describe the arts and cultural sector (or any sector, mind you). In a time of pandemics, climate change, and other reckonings, we might want to ask just how useful our five-year plans really are.
In my experience, the organisations which thrived during COVID-19 were those who were willing to hold their strategic plans lightly and focus on outcomes for artists and audiences. If you planned to tour three works in 2019, and four in 2020, you were in for a rude awakening as border closures would’ve rendered that KPI unattainable. However, if you were pursuing the outcome of equity of access for regional audiences, you may have been able to shift to digital delivery — missing the KPI but hitting the spirit of it in the process. The good, flexible evaluation framework enables this kind of focus on outcomes and allows us to strategically adjust our activities to meet them.
I think it is unlikely that the sector will move beyond strategic planning any time soon. Given that, a good evaluation framework is best considered the cool-headed friend that keeps our strategic plans in check. The strategic plan focuses on our aspirations for the future, and holds us accountable to the work that we claim we will do. The evaluation framework stores our successes from the past, and helps tell whether or not our work led to real change for our artists, audiences, and communities. And, when things really do go wrong — when the strategy doesn’t go according to the strategic plan — it may actually be the evaluation framework that is most helpful. The lean, good-enough framework reminds us what we truly care about and those who our work is intended to serve, and a reminder of those priorities will ensure our strategies remain relevant.