Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

Abstract

Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this paper, we consider the MG as a multi-agent environment with IoT devices in which AI agents exchange information with their peers for collaboration. However, the collaboration information may be lost due to communication failures or packet loss. Such events may affect the operation of the whole MG. To this end, we propose a multi-agent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures. We first define a multi-agent partially observable Markov decision process (MA-POMDP) to describe agents under communication failures, in which each agent can update its beliefs on the actions of its peers. Then, we apply a double deep Q-learning (DDQN) architecture for Q-value estimation in BA-DRL, and propose a belief-based correlated equilibrium for the joint-action selection of multi-agent BA-DRL. Finally, the simulation results show that BA-DRL is robust to both power supply uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3% higher reward than Nash Deep Q-learning (Nash-DQN) and alternating direction method of multipliers (ADMM) respectively under 1% communication failure probability.

Grafik Top
Authors
  • Zhou, Hao
  • Aral, Atakan
  • Brandic, Ivona
  • Erol-Kantarci, Melike
Grafik Top
Projects
Grafik Top
Shortfacts
Category
Journal Paper
Divisions
Scientific Computing
Subjects
Datenverarbeitungsmanagement
Kuenstliche Intelligenz
Angewandte Informatik
Rechnerperipherie, Datenkommunikationshardware
Parallele Datenverarbeitung
Systemarchitektur Allgemeines
Journal or Publication Title
IEEE Internet of Things Journal
ISSN
2327-4662
Publisher
IEEE
Date
1 December 2021
Export
Grafik Top