In this blog post we will explore how SAP BTP can support the current transformation the Utilities sector is undergoing, by implementing AI solutions to enhance communication between energy providers and prosumers.
In this blog post, we will introduce the energy and utilities industry and we will discuss its challenges and the pain points. After that we will see how to design a solution architecture that can address some of these challenges. For those who are interested, we will discuss how the components of the solution can be implemented in a separate blog post that you can find here.
For the comprehension of the content in both the blog posts, it would be good if you were already familiar with some SAP products, in particular SAP BTP platform and SAP AI Core, and if you have already some knowledge about machine learning and data science and also a basic knowledge about the Internet of Things.
Please, note that these blog posts are part of a series of technical enablement sessions on SAP BTP for Industries. Check the full calendar here to watch the recordings of past sessions and register for the upcoming ones! The replay of the session related to these blog posts and all the other sessions is available here.
Authors: amagnani, jacobtan, cesare.calabria
The energy sector is nowadays undergoing a huge structural transformation and we can expect that in a couple of decades the energy grid will operate very differently than today. There are at least two important reasons for this transition. First, the need to fight energy poverty. We know the global population is constantly increasing and that there are 750 million people in the world with no access to electricity, so the energy production must increase to give everyone a chance to benefit from it.
On the other hand, we are facing the challenge of global warming. According to the Paris Agreement, we must keep the global temperature rise below 1.5° C to avoid the most severe consequences. And we know energy production is the first responsible for the emissions of greenhouse gases. In order to have a chance to meet this goal, we must reach net zero carbon emissions by 2050, which means reshaping the way we produce energy.
There are three main fronts on which action must be taken to achieve these net zero carbon emissions by 2050. The first one concerns renewable energy sources. We must ensure that almost 90% of global electricity generation comes from renewable energy sources by 2050, of which solar, photovoltaic and wind together have to account for almost 70%.
We need also to take care of energy efficiency: we need to introduce massively energy efficient solutions for buildings, vehicles, home appliances and also for the industry. And as electricity generation becomes cleaner, we can expect that areas previously dominated by fossil fuels will be electrified, and this can take place with technologies like electric cars, buses and trucks on the roads, heat pumps into buildings and electric furnaces for steel production. By 2050 it is necessary for 100% of two-wheelers to be electric and the same for at least 80% of cars.
In order to integrate the renewable energy sources and for example, to integrate the renewable energy sources, the energy grid is transitioning from a centralized grid based on few large power plants, mostly based on fossil fuels, to a local decentralized grid where we are able to embrace these renewable sources such as wind or solar with many smaller energy producers, with consumers that participate in the energy production with solar panels on the rooftops (Fig. 1).
As usual, there are many challenges in this transition journey. The first one is that the renewable energy sources are by nature intermittent. Another important issue is that at the moment electricity cannot be stored in large amounts and then supply and demand must always be matched or balanced by system operators. And there is always there is often a lack of coordination between consumers and energy operators that leads, for instance, to activating energy loads at suboptimal times resulting in high demand or to an excess of energy production from non-renewable sources. Moreover, in some cases the lack of simple of a simple automated compensation mechanism has discouraged the installation of renewable systems.
How can we address these issues? We need to develop a smart energy grid, a smart energy grid that is able to:
And how can we achieve this smart energy grid? We can leverage artificial intelligence that will power all the tasks above. So let's see how we can design a solution to achieve this AI embedded flexible energy grid.
We've seen that among the challenges that we are experiencing in the energy sector, there is this difficulty in the coordination between the energy production and the energy consumers or prosumers. How can we imagine this coordination and communication happening? Let's refer to Fig. 2.
We can imagine that on one side we have Mary, an energy grid operator that everyday needs to understand how much energy is going to be required tomorrow, for instance for the household population in a certain region, in order to understand if there would be simple criticalities, if the energy demand is going to be matched by the supply or not and take actions accordingly. On the other end, we have John who is a prosumer and needs to know at what time, for example, it's best to do the laundry or at what time it's best to charge his electric car without overloading the grid and at the same time minimizing his energy bill. So how can we help them?
So, on one side, we need to be able to forecast the energy supply that the provider is able to dispatch for tomorrow. On the other side, we need also to forecast what's the energy that John is going to be using tomorrow. And actually not only John, but we need to know also this energy demand of course for all his neighbours or all the consumers in a certain region. Once we have these two inputs, we can imagine to provide Mary with a software that is able to optimize the energy distribution in the neighbourhood and that allows to balance, if needed, the demand and supply to give priority, for instance, to critical infrastructures such as hospitals in case there is a lack of energy or a high demand peak. And once the energy grid operator has decided how much energy is going to be dispatched in the following day, this information can be communicated back to John, the prosumer, so that he can optimize his usage of electric appliances and, for instance, charge the electric car in an hour when the energy grid is not going to be overloaded.
Let's have a look in particular at the prosumer side. How does this work? On the prosumer side (Fig. 3), we can imagine that John has several appliances at his place and let's imagine he has also some solar panels on his rooftop. The heart of our solution is the smart meter.
The smart meter technologies are available today in the market, though not every one of us has smart meter installed at home, but we will see smart meters becoming more and more popular in the coming years. The smart meters can measure the power consumed by individual appliances attached to the electrical circuits with a very high time granularity. And this kind of data can be used to help forecasting the John's energy demand and they can also be used to monitor the energy power produced by the John's solar panel. So these data, the energy produced by the solar panels and the consumption of single appliances at John's house can be used to forecast his energy demand, and they can also be enriched with external data sources, for instance weather data, to make the forecast more accurate.
Smart meters can also greatly help in the optimization of the electrical appliances at prosumer’s house because this technology enables to switch any deferrable loads on and off depending on what's the optimal time of usage. For instance, they can start automatically the laundry or the charging of an electrical, an electric car when it is deemed to be to be optimal.
Now if we go back to the full picture of this coordination between Mary and John, you can recognize that there are many implementations of AI in the story. We mentioned a couple of forecasting algorithms, the forecast of the energy supply, the forecast of the energy demand. There are also a couple of optimization processes that need to run, the optimization of the energy distribution and the organization of the usage of the electrical appliances on the other side. In this use case, we will not talk about all these pieces, but we will focus especially on one little piece, which is the forecasting of the prosumer energy demand for the following day. Now let's see how we can design a solution for that with SAP BTP.
As mentioned earlier, we want to utilize SAP BTP to build a prototype solution that produces a daily forecast of the energy demand at the level of single prosumers. Thus, we have the following three main requirements for this solution:
From the requirements above, it is clear that we need to integrate Artificial Intelligence into an IoT architecture. There are two possible strategies to do that: go with an IoT traditional architecture or with an IoT Edge architecture. This is true not only in our energy grid scenario, but for any IoT application that requires artificial intelligence. Let's discuss both the approaches.
The traditional IoT approach (Fig. 4), characterized by its simplicity and cost-effectiveness, relies on a centralized architecture where smart meters communicate directly with SAP Cloud for Energy (C4E). C4E serves as an IoT hub for energy and water meters, validating and storing meter data. This consolidated data then forms the foundation for demand forecasting models.
Leveraging SAP AI Core, demand forecast models are developed, incorporating historical energy consumption data and weather data retrieved from external APIs. These models generate comprehensive forecasts that are subsequently presented to end users through SAP Analytics Cloud or a customized energy planner application.
The traditional scenario described earlier on, however have some pros and cons: in C4E it is currently possible to store the household consumption at the rate of a measurement of like 15 minutes, while nowadays smart meters have the possibility to measure consumption of each individual appliance with high granularity (one measurement per second). So potentially the granular data coming from the smart meters can help to produce a more accurate forecast of the prosumer energy demand.
However, this would be too much data to be stored entirely in the cloud. So if we want to leverage this granular data for the demand forecast, we would need to actually opt for an Edge IoT solution. The Edge IoT approach (Fig. 5), embraces a decentralized architecture, shifting the computational burden to the smart meters themselves. This strategic placement enables the utilization of granular smart meter data, but at the same time, it introduces architectural complexities. To effectively manage the smart home device runtime and facilitate seamless communication between SAP BTP and the devices, a third-party IoT solution becomes an indispensable component.
There is no right and wrong choice between the implementation of the IoT traditional strategy versus the edge architecture. The decision hinges on the specific requirements of the energy provider. If granular smart meter data is paramount for achieving unparalleled forecasting accuracy, then Edge IoT emerges as the preferred choice. However, if simplicity and cost-effectiveness are prioritized, then traditional IoT may be a more suitable solution.
The capabilities of the Edge IoT architecture extend beyond demand forecasting, offering a platform for fostering enhanced communication between energy providers and consumers (Fig. 6). This enhanced communication can encompass:
In our use case we embarked on a journey to develop a prototype Edge IoT solution, utilizing Azure as the third party IoT platform of choice. Based on the requirements set forth, we focused on crafting the demand forecast model, deploying it to the edge, and establish seamless communication between BTP and the edge device. Of course, as you may argue Azure is not the only sole viable option, but any other third party IoT platform be used as well.
In the video linked below you can find an overview of the solution and the services that we have implemented in our prototype.
In this blog post we have learned how critical is to reach the goal of net zero carbon emission by 2050 and the difficult transition the energy and utility sector is going through. And we've seen that SAP BTP provides all the tools to develop a flexible energy grid powered by artificial intelligence that is able to achieve the integration of the renewable energy sources and enhanced the coordination between between the supply and the demand. Furthermore, we have seen that SAP BTP can be integrated with any third party IoT platform to allow the communication between each household and a central cloud controller handled by the utility provider. If you want to know more about the technical implementation of our prototype, jump to the second blog post.
In this blog post, we will introduce the energy and utilities industry and we will discuss its challenges and the pain points. After that we will see how to design a solution architecture that can address some of these challenges. For those who are interested, we will discuss how the components of the solution can be implemented in a separate blog post that you can find here.
For the comprehension of the content in both the blog posts, it would be good if you were already familiar with some SAP products, in particular SAP BTP platform and SAP AI Core, and if you have already some knowledge about machine learning and data science and also a basic knowledge about the Internet of Things.
Please, note that these blog posts are part of a series of technical enablement sessions on SAP BTP for Industries. Check the full calendar here to watch the recordings of past sessions and register for the upcoming ones! The replay of the session related to these blog posts and all the other sessions is available here.
Authors: amagnani, jacobtan, cesare.calabria
Introduction
The energy sector is nowadays undergoing a huge structural transformation and we can expect that in a couple of decades the energy grid will operate very differently than today. There are at least two important reasons for this transition. First, the need to fight energy poverty. We know the global population is constantly increasing and that there are 750 million people in the world with no access to electricity, so the energy production must increase to give everyone a chance to benefit from it.
On the other hand, we are facing the challenge of global warming. According to the Paris Agreement, we must keep the global temperature rise below 1.5° C to avoid the most severe consequences. And we know energy production is the first responsible for the emissions of greenhouse gases. In order to have a chance to meet this goal, we must reach net zero carbon emissions by 2050, which means reshaping the way we produce energy.
There are three main fronts on which action must be taken to achieve these net zero carbon emissions by 2050. The first one concerns renewable energy sources. We must ensure that almost 90% of global electricity generation comes from renewable energy sources by 2050, of which solar, photovoltaic and wind together have to account for almost 70%.
We need also to take care of energy efficiency: we need to introduce massively energy efficient solutions for buildings, vehicles, home appliances and also for the industry. And as electricity generation becomes cleaner, we can expect that areas previously dominated by fossil fuels will be electrified, and this can take place with technologies like electric cars, buses and trucks on the roads, heat pumps into buildings and electric furnaces for steel production. By 2050 it is necessary for 100% of two-wheelers to be electric and the same for at least 80% of cars.
In order to integrate the renewable energy sources and for example, to integrate the renewable energy sources, the energy grid is transitioning from a centralized grid based on few large power plants, mostly based on fossil fuels, to a local decentralized grid where we are able to embrace these renewable sources such as wind or solar with many smaller energy producers, with consumers that participate in the energy production with solar panels on the rooftops (Fig. 1).
Figure 1: From a centralized to a distributed energy grid.
As usual, there are many challenges in this transition journey. The first one is that the renewable energy sources are by nature intermittent. Another important issue is that at the moment electricity cannot be stored in large amounts and then supply and demand must always be matched or balanced by system operators. And there is always there is often a lack of coordination between consumers and energy operators that leads, for instance, to activating energy loads at suboptimal times resulting in high demand or to an excess of energy production from non-renewable sources. Moreover, in some cases the lack of simple of a simple automated compensation mechanism has discouraged the installation of renewable systems.
How can we address these issues? We need to develop a smart energy grid, a smart energy grid that is able to:
- predict the energy produced by solar and wind parks, hydroplants etc;
- predict the energy demand at regional level or at consumer level;
- rebalance the inequity between production and the consumption loads.
And how can we achieve this smart energy grid? We can leverage artificial intelligence that will power all the tasks above. So let's see how we can design a solution to achieve this AI embedded flexible energy grid.
We've seen that among the challenges that we are experiencing in the energy sector, there is this difficulty in the coordination between the energy production and the energy consumers or prosumers. How can we imagine this coordination and communication happening? Let's refer to Fig. 2.
We can imagine that on one side we have Mary, an energy grid operator that everyday needs to understand how much energy is going to be required tomorrow, for instance for the household population in a certain region, in order to understand if there would be simple criticalities, if the energy demand is going to be matched by the supply or not and take actions accordingly. On the other end, we have John who is a prosumer and needs to know at what time, for example, it's best to do the laundry or at what time it's best to charge his electric car without overloading the grid and at the same time minimizing his energy bill. So how can we help them?
Figure 2: Coordination steps between the energy production and the energy prosumers.
So, on one side, we need to be able to forecast the energy supply that the provider is able to dispatch for tomorrow. On the other side, we need also to forecast what's the energy that John is going to be using tomorrow. And actually not only John, but we need to know also this energy demand of course for all his neighbours or all the consumers in a certain region. Once we have these two inputs, we can imagine to provide Mary with a software that is able to optimize the energy distribution in the neighbourhood and that allows to balance, if needed, the demand and supply to give priority, for instance, to critical infrastructures such as hospitals in case there is a lack of energy or a high demand peak. And once the energy grid operator has decided how much energy is going to be dispatched in the following day, this information can be communicated back to John, the prosumer, so that he can optimize his usage of electric appliances and, for instance, charge the electric car in an hour when the energy grid is not going to be overloaded.
Let's have a look in particular at the prosumer side. How does this work? On the prosumer side (Fig. 3), we can imagine that John has several appliances at his place and let's imagine he has also some solar panels on his rooftop. The heart of our solution is the smart meter.
Figure 3: The role of smart meter on the prosumer side.
The smart meter technologies are available today in the market, though not every one of us has smart meter installed at home, but we will see smart meters becoming more and more popular in the coming years. The smart meters can measure the power consumed by individual appliances attached to the electrical circuits with a very high time granularity. And this kind of data can be used to help forecasting the John's energy demand and they can also be used to monitor the energy power produced by the John's solar panel. So these data, the energy produced by the solar panels and the consumption of single appliances at John's house can be used to forecast his energy demand, and they can also be enriched with external data sources, for instance weather data, to make the forecast more accurate.
Smart meters can also greatly help in the optimization of the electrical appliances at prosumer’s house because this technology enables to switch any deferrable loads on and off depending on what's the optimal time of usage. For instance, they can start automatically the laundry or the charging of an electrical, an electric car when it is deemed to be to be optimal.
Now if we go back to the full picture of this coordination between Mary and John, you can recognize that there are many implementations of AI in the story. We mentioned a couple of forecasting algorithms, the forecast of the energy supply, the forecast of the energy demand. There are also a couple of optimization processes that need to run, the optimization of the energy distribution and the organization of the usage of the electrical appliances on the other side. In this use case, we will not talk about all these pieces, but we will focus especially on one little piece, which is the forecasting of the prosumer energy demand for the following day. Now let's see how we can design a solution for that with SAP BTP.
Solution architecture
As mentioned earlier, we want to utilize SAP BTP to build a prototype solution that produces a daily forecast of the energy demand at the level of single prosumers. Thus, we have the following three main requirements for this solution:
- access energy consumption data captured by smart meters;
- enrich and integrate with additional data sources to achieve accurate forecast;
- train and deploy forecasting machine learning models.
From the requirements above, it is clear that we need to integrate Artificial Intelligence into an IoT architecture. There are two possible strategies to do that: go with an IoT traditional architecture or with an IoT Edge architecture. This is true not only in our energy grid scenario, but for any IoT application that requires artificial intelligence. Let's discuss both the approaches.
Traditional IoT architecture: a straightforward approach
The traditional IoT approach (Fig. 4), characterized by its simplicity and cost-effectiveness, relies on a centralized architecture where smart meters communicate directly with SAP Cloud for Energy (C4E). C4E serves as an IoT hub for energy and water meters, validating and storing meter data. This consolidated data then forms the foundation for demand forecasting models.
Leveraging SAP AI Core, demand forecast models are developed, incorporating historical energy consumption data and weather data retrieved from external APIs. These models generate comprehensive forecasts that are subsequently presented to end users through SAP Analytics Cloud or a customized energy planner application.
Figure 4: Traditional IoT – Sample Solution Architecture.
Edge IoT architecture: harnessing the power of edge computing
The traditional scenario described earlier on, however have some pros and cons: in C4E it is currently possible to store the household consumption at the rate of a measurement of like 15 minutes, while nowadays smart meters have the possibility to measure consumption of each individual appliance with high granularity (one measurement per second). So potentially the granular data coming from the smart meters can help to produce a more accurate forecast of the prosumer energy demand.
However, this would be too much data to be stored entirely in the cloud. So if we want to leverage this granular data for the demand forecast, we would need to actually opt for an Edge IoT solution. The Edge IoT approach (Fig. 5), embraces a decentralized architecture, shifting the computational burden to the smart meters themselves. This strategic placement enables the utilization of granular smart meter data, but at the same time, it introduces architectural complexities. To effectively manage the smart home device runtime and facilitate seamless communication between SAP BTP and the devices, a third-party IoT solution becomes an indispensable component.
Figure 5: AI at the Edge – Sample Solution Architecture.
Selecting the optimal approach: a balancing act
There is no right and wrong choice between the implementation of the IoT traditional strategy versus the edge architecture. The decision hinges on the specific requirements of the energy provider. If granular smart meter data is paramount for achieving unparalleled forecasting accuracy, then Edge IoT emerges as the preferred choice. However, if simplicity and cost-effectiveness are prioritized, then traditional IoT may be a more suitable solution.
Enriching the architecture: unleashing the potential of edge IoT
The capabilities of the Edge IoT architecture extend beyond demand forecasting, offering a platform for fostering enhanced communication between energy providers and consumers (Fig. 6). This enhanced communication can encompass:
- employing a second machine learning model to forecast energy supply, enabling energy planners to optimize energy distribution on a granular level;
- empowering energy planners to compare demand and supply forecasts, facilitating informed decisions regarding energy distribution to various prosumers;
- communicating energy availability to smart home devices, allowing prosumers to optimize appliance usage and avoid grid overloading.
Figure 6: Solution architecture highlighting in-scope (green) and out-of-scope (violet) components.
In our use case we embarked on a journey to develop a prototype Edge IoT solution, utilizing Azure as the third party IoT platform of choice. Based on the requirements set forth, we focused on crafting the demand forecast model, deploying it to the edge, and establish seamless communication between BTP and the edge device. Of course, as you may argue Azure is not the only sole viable option, but any other third party IoT platform be used as well.
In the video linked below you can find an overview of the solution and the services that we have implemented in our prototype.
Key takeaways
In this blog post we have learned how critical is to reach the goal of net zero carbon emission by 2050 and the difficult transition the energy and utility sector is going through. And we've seen that SAP BTP provides all the tools to develop a flexible energy grid powered by artificial intelligence that is able to achieve the integration of the renewable energy sources and enhanced the coordination between between the supply and the demand. Furthermore, we have seen that SAP BTP can be integrated with any third party IoT platform to allow the communication between each household and a central cloud controller handled by the utility provider. If you want to know more about the technical implementation of our prototype, jump to the second blog post.
Links
- POC GitHub repository: https://github.com/SAP-samples/btp-industry-use-cases/tree/main/08-ai-embedded-flexible-energy-grid
Utilities industry
- Net Zero Roadmap: A Global Pathway to Keep the 1.5 °C Goal in Reach IEA Report
- Harnessing AI to accelerate the Energy TransitionWorld Economic Forum White Paper 2021
- How the grid is becoming more human-centricPodcast
- https://i-nergy.eu/ European Project to fund AI use cases related to sustainable energies
- State-of-the-Art Artificial Intelligence Techniques for Distributed Smart Grids: A Review Research Paper
- Watt’s up at Home? Smart Meter Data Analytics from a Consumer-Centric Perspective Research Paper
- AMPds The Almanac of Minutely Power Data Dataset
- Electricity forecasting on the individual household level enhanced based on activity patterns Research Paper
More about BTP integration with third-party IoT platforms
- Events-to-Business Actions”: An event-driven architecture on SAP BTP to implement Industry 4.0 scena...
- Build Events-to-Business Actions Apps with SAP BTP and MS Azure/AWS
SAP AI Core and AI Launchpad
SAP Datasphere
SAP Analytics Cloud
SAP Cloud for Energy
- SAP Managed Tags:
| User | Count |
|---|---|
| 19 | |
| 14 | |
| 11 | |
| 10 | |
| 9 | |
| 8 | |
| 7 | |
| 7 | |
| 7 | |
| 6 |
