Data Really is the New Oil

How Data Took Over the Arts and Polluted Our Judgment

Welcome to Measuring the Immeasurable, a newsletter helping arts and culture leaders understand the language of public value. Firstly, a warm welcome to any new subscribers. It has been great to see this newsletter get some traction in the last month or so.

Given a few new people have subscribed, this is a slightly longer introductory piece. I outline what I see as the main challenges in the way we think about evaluation in the arts. The main way people discover my writing is by others sharing it, so please pass this on to others if you found it useful.

---

Landscape, Cheng Tinglu. Open Access from The Met.

The simile ‘Data is the New Oil,’ first appeared in The Economist in 2017¹, and referred to the emerging industry of making sense from enormous quantities of bits and bytes. In the private sector, it captured the excitement around big data and machine learning. In the public sector, it spoke to the new powers of governments to more efficiently deploy resources in alignment with data-driven insights.

The intention of the simile was one dimensional: To denote that data had become profoundly valuable. However, I would extend the simile a little further to say that data is also like oil in that we are:

1. Addicted to it (no amount of data satisfies us).

2. Failing to account for the impacts in extracting, transporting and using it.

3. Failing to use it wisely.

We are Addicted to Data

In the arts, the collection of quantitative data has become a bit of theatrical pomp. Data is collected and provided to funding agencies largely to give the illusion of diligence and rigour. Funding agencies request data for largely the same reasons. In government, decisions must be seen to be based on evidence rather than fallible human jugdement.

The problem is that data never provides a complete-enough picture to make judgment straightforward. Rather than admit uncertainty and exercise judgment, agencies respond by seeking even more data. As arts organisations’ work becomes more complex (education, digital engagement, advocacy, diversity and inclusion, and so on), the expectations around data collection and reporting also become more complex. This data then raises additional questions, which drives even more data collection. This is a neverending cycle of addiction.

The best examples of this are the impact reports of major festivals, which are awash in quantitative data. Want to know how many middle-aged white women saw theatre in Gosnells? Want to know how many audience respondents assessed contemporary dance productions within the fifth quintile of risk? It’s probably in the data. If you want to know why middle-aged women are seeing theatre in Gosnells, or what kind of production might work there next year, you will be out of luck. Data is a no substitute for storytelling, discernment, and judgment. Ironic that it should happen in the arts of all sectors.

T. S. Eliot may have been talking about our addiction to data, and the way it has come to replace judgment, when he wrote of those who:

“constantly try to escape
From the darkness outside and within
By dreaming of systems so perfect that no one will need to be good.”

We Fail to Account for the Costs of Collecting Data

Evaluation comes at a considerable cost to arts administrators and funders. Just as the pollution caused by oil is not adequately accounted for, government and philanthropic bodies rarely account for the administrative overheads involved in grant applications, acquittals and evaluations². Creating and communicating public value, outcomes measurement, and program analysis and evaluation were all identified as deficit areas across a survey of arts organisations, along with evaluation-adjacent areas like grant writing³.

When a funder asks for a report or acquittal, this is by extension a demand on an organisation to improve its capacity to evaluate. This isn’t inherently a bad thing. I believe arts organisations should be digitally capable and collecting data. However, when this work is not costed and takes place within the grant churn cycle, it rarely leads to capacity building for arts organisations.

We Don’t Use Data Wisely

Most arts organisations consider their data collection and reporting requirements a burden. However, that burden also exists on the other side of the ledger. Everything they collect must be analysed. Data is a debt before it is an asset.

Some sense must be derived from dozens of productions, across myriad artforms, all providing scores on ‘excellence’ or ‘risk.’ Given the sheer volume of quantitative data and the lack of qualitative context which surrounds this data, making sense of it is an impossible task. Deriving meaning from carelessly collected quantitative data is like trying to write an award winning novel by dumping a can of alphabet soup on the kitchen floor.

Sustainable Data

The alternative to our current status quo is sustainable data. This means collecting a small, carefully discerned amount of qualitative and quantitative data which has meaning for both organisations and stakeholders. I usually suggest starting with stories (or even just testimonials), which are a rich form of qualitative data that most arts organisations are already collecting in some form or another⁴. Then, a sensible set of highly relevant quantitative data can be collected which supports this quantitative data.

This approach, done properly, is not only sustainable but regenerative.

1. The Economist, 2017.

2. See Paying What it Takes, Centre for Social Impact and Social Ventures Australia.

3. See Sector Audit Report, Chamber of Arts and Culture WA.

4. See You’re Already Evaluating Your Work, an earlier post from this Newsletter, for a bit more on this approach.