Broken data-flows are wasting all our time & inhibiting innovation

Gavin Starks
5 min readFeb 10, 2020

--

“Open up the data”

“If only we had the data we could make better decisions”

“We should put all the data in one place so that we can run our [analytics/AI/ML/etc] to work out ‘the answer’”

I’ve spent decades hearing calls like these from governments, global enterprises, corporates, NGOs, charities, small companies… but to what end? Over the last decade we’ve seen huge investments in the creation of vast data lakes that, largely, seem to end up stagnant; filled with unrealised potential.

Here we’re going to go right into the frontline: what are we trying to communicate in an environmental report? And why? What behaviours are we trying to change? What material decisions are being made?

Let’s explore a common scenario. Imagine you work at a large, global enterprise with many facilities. Your role is to report on ‘CSR’ or ‘ESG’ related to environmental impacts. The process of data gathering is complex: some data will be gathered via billing systems, BIM systems, sensors, utility providers and aggregators. In some cases, part of the data acquisition may be via data management companies and systems integrators. Some may even be paper-based reports — there are over 50,000 water utilities and over 3,500 electricity companies in the US alone.

Regardless of the size, technical competence or maturity of an organisation, I have yet to find a single example where the process of collating, cleaning and aggregating the data required to create an environmental report does not require manual intervention in a spreadsheet.

And, once this work is collated, checked and put into an annual report, it ends up in an unstructured form, often in a PDF. This is in Y2020, after a decade of ‘big data’, the web, APIs.

To paraphrase one story:

“We’re always sprinting so hard to the next reporting deadline or trying to use the data we have to create some kind of impact, so we never get round to updating the way we collect and process it — like an old car that you know is on its last legs, but you know how to start it, so it’s good enough”.

It may feel like that big companies have it all in-hand: some even have internal pricing on carbon. But this doesn’t mean that the process of measurement is resolved.

And then what happens? Well, various ‘market intelligence companies’ and indexes scrape the data out of the PDFs and put them within reach of the markets and investors. The market can compare some elements of the reports, and do some kind of comparison with competitors. Does this drive investment behaviour? Barely.

Some investors who have more of a priority on ESG but really, what are they comparing? The sentiment I’ve heard is “mostly screening”: do you have a CSR report? Do you track CO2? Of the 10,000 companies we are looking at, are you in the top 10%? And the people actually doing this work are usually the junior analysts, trying to impress their boss.

So, the decisions often end up as ‘how involved are they in oil and gas?’.

“Ah”, I hear you say, “but there are other parts of the system that do far more detailed modelling. They have much more accurate insights.”

Yes. And no.

Yes, insurance companies have highly detailed models of risk. And, these are mostly ‘black boxes’ into which data is piped and risk analysis is output (not to say they are not accurate, but how do we know what science, scope and details are incorporated in the models?). But before we even get into that, I hear stories of junior members of staff being emailed 12,000 self-reported spreadsheets a year, which are keyed into systems multiple times, then passed on and data re-keyed and re-keyed. It can’t be fun, and I somewhat doubt it is flawless.

I’ve heard stories of asset managers struggling to bring together disparate data sets to even correctly identify precisely where the assets are.

So where does this leave us?

The promise of the industry-led Green Button Initiative in the US seems to have waned? Yet in banking, we have the remarkable cohesion of the regulated Open Banking Standard in the UK — and calls to ‘repurpose it’ to not only Open Finance but to the energy and other sectors.

We also have calls to take material steps forward in Climate-related Financial Disclosures which is essential, and I can’t help but think that such initiatives must address common and shared principles and practice for data sharing to avoid the vast inefficiencies we see in the current data ecosystems.

At the same time, we have huge advances in planetary-scale data flow.

Earth observation has reached a point where we can have regular updates of maps of the majority of the planet, at three-metre resolution (about the size of your car).

Infrastructure and engineering firms are instrumenting assets, infrastructure and cities: there is no reason why we couldn’t design assets to self-report to the broad ecosystem of users that would like their insights. Perhaps the promise of the ‘smart city’ or ‘internet of things’ has finally found a purpose?

But, with all this clear opportunity to radically increase our data-flows, its timeliness and its resolution, we still need to sort out the principles, processes and practices of how to make it work in a sustainable and scalable manner for both humans and machines. Otherwise, we’ll still be driving our old clapped-out spreadsheets when we should be driving a Tesla.

And, we need to evolve our decision-making — quickly — to deliver net-zero outcomes. It often feels like we lose sight of the wood for the trees because we’ve not manifest the right culture for change.

If the 300+ conversations I’ve had over the last year represents the state-of-the-nation, how are we going to evolve credible financial instruments such as Climate Bonds (which are fixed-income financial instruments linked ‘in some way’ to climate change solutions) to be demonstrably linked to outcomes in a manner that is accountable? How are we going to change the behaviours of asset owners and markets, pension funds, insurers, lenders and investors?

How will we turn all this data flow into trusted, demonstrable, auditable and actionable operational information that drive our business decisions now (this year)?

At Icebreaker One we are exploring real-world use-cases that can be used to illustrate the potential to align policy, science, finance and business; identify opportunities and gaps; map the ecosystem and supply-chains; create a roadmap to measurable impact.

SERI is our first substantial project in this area — please get in touch if you are interested.

--

--