Real-time security margin control using deep reinforcement learning

Journal article, 2023

This paper develops a real-time control method based on deep reinforcement learning aimed to determine the optimal control actions to maintain a sufficient secure operating limit. The secure operating limit refers to the limit to the most stressed pre-contingency operating point of an electric power system that can withstand a set of credible contingencies without violating stability criteria. The developed deep reinforcement learning method uses a hybrid control scheme that is capable of simultaneously adjusting both discrete and continuous action variables. The performance is evaluated on a modified version of the Nordic32 test system. The results show that the developed deep reinforcement learning method quickly learns an effective control policy to ensure a sufficient secure operating limit for a range of different system scenarios. The performance is also compared to a control based on a rule-based look-up table and a deep reinforcement learning control adapted for discrete action spaces. The hybrid deep reinforcement learning control managed to achieve significantly better on all of the defined test sets, indicating that the possibility of adjusting both discrete and continuous action variables resulted in a more flexible and efficient control policy.