• Pete Clutton-Brock

Data discovery and access key to achieving net zero

Across the sectors that matter when it comes to addressing climate change, there is an opportunity to apply data science and AI to accelerate progress. Take the electricity sector for example. As we get more renewable energy on the grid we are going to need to focus on how to optimise the balancing of renewables. Successful balancing will require thousands, potentially hundreds of thousands of energy storage and demand-side-response assets to be connected to the grid. To ensure that all of these assets are optimised and collectively achieve their goal of balancing the system there will be a need for them all to be optimised using intelligent software. The Centre for AI & Climate has written previously about the need for AI-optimised electricity markets. However, to enable these companies' AI algorithms to work effectively it will be critical to ensure that there is much greater data visibility across the system, such that any one part of the system can understand the wider context in which it is offering services. This will require a step change in discovery of and access to energy-related data. For example developers of balancing assets will need a better understanding of where there are constraints on the grid now and where there are likely to be risks of supply and demand imbalances in the future. Developers of AI trading algorithms for energy assets would be able to offer a step change in effectiveness if there was more data on current and forecast supply and demand across the networks. EV charging infrastructure developers need visibility of where they can connect their assets to networks at lowest possible cost. DNOs need better visibility of distributed energy resources connected to their networks. The system operator needs much better forecasting of supply and demand to optimise the final stage of balancing through the Balancing Mechanism and ancillary markets. Suppliers offering agile tariffs will need greater visibility of current and forecast network constraints to understand where to focus the roll-out of such services. These are just a small sample of how improved data access will be critical for the transition to a net zero grid, but there will be thousands of opportunities created if data is made more available.


Beyond energy the same is true for other sectors such as transport, land use and industry.


For data scientists working on climate change, one of the most significant time sinks, is simply finding the right data as it is often held by a myriad of different data providers.


If we could simply aggregate links to climate transition data, we could radically reduce the time overhead data scientists experience in finding data. At a basic level this could be a simple online, open-access data catalogue, of links that connect data scientists to already-open data that can be found online. Beyond this initial approach however, to fully enable optimal data discovery, a data platform is needed that can support the following evolutions to support net zero-relevant data access:

  • From individual datasets to an interconnected data graphs that captures the relationships between different datasets;

  • From basic to increasingly in-depth data context to allow those coming to the field fresh to understand the data landscape easily;

  • From a limited number of datasets to a very large number of datasets, as the system is increasingly well monitored;

  • From static datasets to a combination of historic data and live data streams via APIs: to support real time system monitoring;

  • From high-level metadata standards to increasingly detailed data guidelines: to allow common datasets to be machine-readable and easily discoverable;

  • From lists of datasets to geospatial visualisations of datasets: because many of the questions that need answering on system optimisation are geospatial in nature;

  • From a system that accepts limited quality data to one with increasing standards for cleaned and labelled datasets;

  • From showcasing open data to facilitating a market for commercial energy data, to allow data owners to recover some of the costs from collecting data.

A platform that enabled these transitions would fully address the problem of data discovery and data access for climate relevant data.


It is clear we will need optimal data discovery to support a rapid transition to net zero. Such a platform would help unleash a Cambrian explosion of digital and AI innovation in support of the transition to net zero and climate resilience. CAIC is considering carefully how to develop such a platform.

Peter Clutton-Brock (@pcbrock) is co-founder of the Centre for AI & Climate.