Welcome to Measuring the Immeasurable, a newsletter helping culture leaders understand the relationship between culture and public value. Quite a few readers have joined in the last fortnight. Welcome, and this piece is as good a place to begin as any. It explores an alternative, more expansive model of the truth that can help cultural change professionals better talk about their work in a way that matters.
As always: if you also find it useful, please consider sharing and subscribing.
Werner Herzog’s The Cave of Forgotten Dreams, a documentary about the oldest cave art in the world, concludes with a rather unusual postscript. The final shot moves from the caves to a nearby greenhouse where several albino crocodiles are kept. Herzog intimates, through his signature narration and a few juxtaposed shots, that the crocodiles’ albinism was caused by the radiation from a nearby nuclear powerplant. The postscript is classic Herzog: suggestive rather than explicit. He invites us to contrast the deep, still time of the cave paintings with the vertigo-inducing impacts of modern life.
Great ending, but there is a catch: Herzog lied to us. The crocodiles’ albinism was a natural mutation completely unrelated to the power plant. On The Colbert Report, he described the final scene as a “wild science-fiction fantasy.” He then claimed his right, as a documentary filmmaker, to indulge in this sort of fantasy. What I found particularly interesting was the language he used. He felt that his job, as a documentary filmmaker, was not to share the “accountant’s truth,” but rather the “ecstatic truth.” He is worth quoting in full here:
“I want the audience with me in wild fantasy in something that illuminates them. You see if I were only fact based, the book of books in literature then would be the Manhattan phone directory — four million entries, everything correct… I am not this kind of a filmmaker.”
In my last instalment on Bad Faith Evaluation, I wrote about how evaluators often seek to model themselves as ‘social accountants.’ This view is gaining traction in important policymaking circles, like the UN1 and Social Value International2. What happens here has a significant influence on how the value of the arts is conceptualised and assessed. In my professional experience, evaluation practice in the arts often mirrors this pursuit of the accountant’s truth. The accountant is obsessed with narrow metrics, completeness, objectivity, and accountability. The accountant describes how a meal tastes by reading the nutrition label.
Cultural organisations, in this milieu, fear being honest. In a world where the accountant’s truth reigns supreme, it is hard to speak truthfully about anything which can’t fit on a spreadsheet. This leads to a sense that evaluation must necessarily be in the language of the accountant, and therefore false or dishonest, if it is to be understood. The tension here is not between what is true and what is false, but between different kinds of truth. Rather than pursue the “accountant’s truth,” cultural evaluation should pursue Herzog’s “ecstatic truth.”
At this point, everyone may start to feel a bit uncomfortable. Are we permitted to lie? Here, I believe, we need our intuitions to guide us. As an evaluator, I do not think I could ever be as egregious as Werner Herzog was in The Cave of Forgotten Dreams. However, the documentary filmmaker shares the same basic challenge that an evaluator does: We are storytellers first, and stories involve crafted truths. Stories always involve omission, composition, and suggestion, and yet they use these tools in the pursuit of a truth that cannot be apprehended by factual reporting alone. I could not lie like Herzog, but I can learn from him.
There is no algorithm, method, or standard that can resolve the tension between truthtelling and storytelling. In every evaluation I write, there is always an audience in mind, and this audience shapes what is written, what is emphasised, and what is excluded. Far from diminishing the credibility of evaluation, my experience is that reports written with a spirit of storytelling tend to be more honest, succinct, and impactful. As Herzog suggests, the ‘phonebook’ evaluation, in trying to tell us everything, tells us nothing of value. The truthful evaluation, we could say, is one where the best of my understanding is offered in service of your understanding.
See SDG Impact.