Deep Reinforcement Learning Based Grid-Forming Inverter
Paper in proceeding, 2023

This research work introduces Deep Deterministic Policy Gradient (DDPG), a type of Reinforcement Learning (RL), for grid modeling, estimating voltage and phase angle, and control method for grid-forming inverters. The aim is to develop a grid-forming inverter that sets the voltage level and frequency of the grid and mitigates voltage dips originating from faults and frequency deviation fluctuations. Unlike conventional methods for estimating setpoints for the controller loops, we do not need several chains of estimation tools such as Fast Fourier Transform (FFT), Synchronous Reference Frame (SRF), or lowpass filters. With the DDPG, we also optimize the phase lock-loop (PLL) and accurately deliver the angle for the actuation part of the inverter to generate the given reference signal. The developed method does not need exhaustive tuning of parameters such as coefficients of PID controllers and lowpass filters. We observe that the proposed method has a faster response time than the PID-based control unit (15ms compared to 50ms) for the grid-forming inverter in the case of compensating voltage dips. We also observed that the DDPG-based grid-forming inverter is more efficient in compensating continuous voltage variations and frequency deviations than a trivial PID-based version.

Voltage fluctuation

PID Controller

Cable fault

Deep Deterministic Policy Gradient

Voltage dip

Frequency Deviations

Reinforcement Learning

Grid-forming Inverter

Author

Ebrahim Balouji

Energy Conversion and Propulsion Systems

Karl Bäckström

Network and Systems

Tomas McKelvey

Chalmers, Electrical Engineering, Signal Processing and Biomedical Engineering

2023 IEEE Industry Applications Society Annual Meeting, IAS 2023


9798350320169 (ISBN)

2023 IEEE Industry Applications Society Annual Meeting, IAS 2023
Nashville, USA,

Subject Categories

Telecommunications

Control Engineering

Other Electrical Engineering, Electronic Engineering, Information Engineering

DOI

10.1109/IAS54024.2023.10406664

More information

Latest update

3/13/2024