Real-time Security Margin Control Using Deep Reinforcement Learning

Preprint, 2021

This paper develops a real-time control method based on deep reinforcement learning (DRL) aimed to determine the optimal control actions to maintain a sufficient secure operating limit (SOL). The SOL refers to the limit to the most stressed pre-contingency operating point of an electric power system that can withstand a set of credible contingencies without violating stability criteria. The developed DRL method uses a hybrid control scheme that is capable of simultaneously adjusting both discrete and continuous action variables. The performance is evaluated on a modified version of the Nordic32 test system. The results show that the developed DRL method quickly learns an effective control policy to ensure a sufficient SOL for a range of different system scenarios. The impact of measurement errors and unseen system conditions are also evaluated. While the DRL method manages to achieve good performance on a majority of the defined test scenarios, including measurement errors during the training phase would improve the robustness of the control with respect to random errors in the state signal. The performance is also compared to a conventional look-up table control where the advantages of the DRL method are highlighted.