Energy management system for microgrids based on deep reinforcement learning

Abstract

The increasing use of distributed and renewable energy resources poses a challenge for traditional control methods. This happens due to the higher complexity and uncertainty introduced by these new technologies, specially in smaller self sufficient systems like microgrids. To address this challenge, reinforcement learning algorithms are used to design and implement an energy management system (EMS) for different microgrids configurations. Reinforcement Learning (RL) approaches seek to train an agent from their interaction with the environment rather than from direct data, such as in supervised learning. With this in mind, the problem of energy management is defined in a Markov decision process and it is solved using different state-of-the-art Deep Reinforcement Learning (DRL) algorithms, such as Deep Q-Networks (DQN), Proximal Policy Optimization (PPO) and Twin Delayed Deep Deterministic Policy Gradient (TD3). Additionally, these results are compared with traditional EMS implementations such as Rule-Based and Model Predictive Control (MPC), used as benchmarks. Simulations are run with the novel Pymgrid module build for this purpose. Results show that DRL EMS agents have comparable results to some the classical implementations with possible benefits for generic and specific use cases. Source code of this project can be found at: https://github.com/Cesard97/DRL-Microgrid-Control