As we get more renewable energy on the grid we are going to need to focus on how to optimise the balancing of renewables. Successful balancing will require thousands, potentially hundreds of thousands of energy storage and demand-side-response assets to be connected to the grid. To ensure that all of these assets are optimised and collectively achieve their goal of balancing the system there will be a need for them all to be optimised using intelligent software. The Centre for AI & Climate has written previously about the need for AI-optimised electricity markets. The outlines of such a system are already being built by forward looking companies. However, to enable these companies' AI algorithms to work effectively it will be critical to ensure that there is much greater visibility across the system, such that any one part of the system can understand the wider context in which it is offering services.
This will require a step change in access to energy-related data. For example developers of balancing assets will need a better understanding of where there are constraints on the grid now and where there are likely to be risks of supply and demand imbalances in the future. Developers of AI trading algorithms for energy assets would be able to offer a step change in effectiveness if there was more data on current and forecast supply and demand across the networks. EV charging infrastructure developers need visibility of where they can connect their assets to networks at lowest possible cost. DNOs need better visibility of distributed energy resources connected to their networks. The system operator needs much better forecasting of supply and demand to optimise the final stage of balancing through the Balancing Mechanism and ancillary markets. Suppliers offering agile tariffs will need greater visibility of current and forecast network constraints to understand where to focus the roll-out of such services. These are just a small sample of how improved data access will be critical for the transition to a net zero grid, but there will be thousands of opportunities created if data is made more available.
How can we rapidly improve energy data access? Luckily the government, having backed the Energy Data Taskforce's recommendations, are supporting this process through the development of a platform to improve energy data discovery, and are tendering for the delivery of an Energy Data Visibility Project.
The initial phase of this project will represent a single point where market participants can easily find existing energy data with common metadata standards. This in itself will be a big step forward, however, it is important to start any project with the end in mind. To fully enable optimal energy data discovery, a successful initial iteration will need to lay the foundations for future iterations. The future of a successful energy data discovery platform will involve the following evolutions to support energy data access:
From individual datasets to an interconnected energy data graph that captures the relationships between different datasets;
From basic to increasingly in-depth data context to allow new market entrants to understand the data landscape easily;
From a limited number of datasets to a very large number of datasets, as the system is increasingly well monitored;
From static datasets to a combination of historic data and live data streams via APIs: to support real time system monitoring;
From high-level metadata standards to increasingly detailed data guidelines: to allow common datasets to be machine-readable and easily discoverable;
From lists of datasets to geospatial visualisations of datasets: because many of the questions that need answering on system optimisation are geospatial in nature;
From a system that accepts limited quality data to one with increasing standards for cleaned and labelled datasets;
From showcasing open data, to facilitating a market for commercial energy data, to allow data owners to recover some of the costs from collecting data.
The realisation of these transitions would lay the foundation for radically improved grid monitoring and allow for simulations of the energy system through which it would be possible to test how to optimise the grid.
The government's Energy Data Visibility Project will need to progress one step at a time. However it is clear we will need optimal energy data discovery to support a net zero grid in 2035. Developing an ambitious approach to data discovery now is not only a critical prerequisite for achieving a net zero grid, but would help unleash a Cambrian explosion of innovation in the energy sector and support the development of a world-leading digital energy technology ecosystem in the UK.
Peter Clutton-Brock (@pcbrock) is co-founder of the Centre for AI & Climate and a senior associate at thinktank E3G.