Abstract

Modern power systems face increasing challenges from renewable energy integration, distributed resources, and complex operational requirements. This survey examines Safe Reinforcement Learning (Safe RL) as a framework for maintaining reliable power system operation while optimizing performance. We review both model-free and model-based approaches, analyzing how different safety constraints and architectures can be implemented in practice. The survey explores multi-agent frameworks for coordinated control in distributed settings and examines runtime assurance methods that provide formal safety guarantees. Applications span various timescales, from frequency regulation to demand management, with different safety requirements and operational contexts. Through analysis of current simulation environments and practical implementations, we identify remaining challenges in scaling safe RL to large power systems, handling uncertainty, and integration with existing infrastructure.

Reinforcement Learning Meets the Power Grid: A Contemporary Survey with Emphasis on Safety and Multi-agent Challenges

The ongoing evolution of power systems presents a multifaceted challenge, namely to ensure a safe and reliable operation amidst a dynamic and uncertain environment. This necessitates not only achieving performance objectives, but also adhering to diverse constraints encompassing operational limits, regulatory compliance, and environmental goals. For example, challenges for modern power systems include renewable energy integration, distributed resources, and complex operational requirements. This monograph examines Safe Reinforcement Learning (safe RL) as a framework for maintaining reliable power system operation while optimizing performance. Both model-free and model-based approaches are reviewed, analyzing how different safety constraints and architectures can be implemented in practice.

The monograph also explores multi-agent frameworks for coordinated control in distributed settings, and examines runtime assurance methods that provide formal safety guarantees. Applications span various timescales, from frequency regulation to demand management, with different safety requirements and operational contexts. Through analysis of current simulation environments and practical implementations, remaining challenges in scaling safe RL to large power systems are identified, as well as handling uncertainty and integration with existing infrastructure.