E-Stream
E-Stream
Project group: E-Stream - Big Data Analytics in the Smart Grid
Our energy grids are becoming increasingly complex. Distributed generators, such as wind farms, solar parks and private PV systems, behave differently to large power plants as they are dependent on the weather, among other things. Added to this is the liberalisation of the energy market, which makes it possible to trade energy on exchanges. Together with energy storage technologies, this ensures that, on the one hand, producers do not necessarily feed energy into the grid when it is generated, but when the energy price is high. On the other hand, consumers can also use energy storage technologies (such as electric cars) and buy energy when it is cheap and not when it is consumed. Consumer behaviour therefore no longer follows standard load profiles.
The monitoring and control of such a highly dynamic grid requires a comprehensive IT infrastructure in which all the data required for grid operation and optimisation is collected, distributed and processed. Such an infrastructure turns an energy grid into a smart grid.
The E-Stream - Big Data Analytics in the Smart Grid project group aims to develop a toolbox that can be used to analyse and visualise the massive amounts of data generated in a smart grid (e.g. from smart meters) in real time. Such tools can make a major contribution to maintaining the stability of our energy grids by calculating the current grid status and forecasting the future grid status in real time. These tools include, for example, short-term forecasts of generator and consumer behaviour based on current data, historical data and weather data. The task here is not to develop new algorithms or data mining methods, but to integrate tried and tested methods. The project group can help decide which tools will ultimately be focussed on.
To enable such real-time analyses, smart meter data is to be simulated with a simulation framework, processed with a data stream management system and visualised with a dashboard, for example.
The open source framework mosaik, written in Python, will be used as the simulation framework. Mosaik makes it possible to combine different simulation models and simulators (in different programming languages) and thus create large-scale smart grid scenarios.
The data stream management system for real-time processing of smart grid data is to be implemented using the Odysseus data stream management framework. Odysseus is also open source, written in Java and based on OSGi. It is therefore possible to create a data stream management system customised for the tasks from a large number of extensions. A core task of the project group will be to extend Odysseus with new functions for data analysis and forecasting of smart grid data and to customise a data stream management system accordingly.
In a final step, the analysis results will then be graphically processed and presented in the form of a dashboard, for example. The project group can decide for itself which technology or frameworks it would like to use to implement this.
Part of the desired toolbox is to be implemented in cooperation with the "Big Data Archive" project group of the VLBA department. The idea is for the raw data from Odysseus to be sent to the Big Data Archive. A data mining model is then learnt there on the long-term data, which can be used by Odysseus to analyse the incoming data in real time. It is important to decouple the two systems appropriately (e.g. using Apache Kafka) so that both project groups can initially work independently of each other.
The project group E-Stream - Big Data Analysis in the Smart Grid is offered together with the Department of Energy Informatics.
- Contact: Michael Brand
- Duration: 1 April 2017 to 31 March 2018 (SS 2017 + WS 2017/2018)