An efficient intrusive deep reinforcement learning framework for OpenFOAM
Journal article, 2024

Recent advancements in artificial intelligence and deep learning offer tremendous opportunities to tackle high-dimensional and challenging problems. Particularly, deep reinforcement learning (DRL) has been shown to be able to address optimal decision-making problems and control complex dynamical systems. DRL has received increased attention in the realm of computational fluid dynamics (CFD) due to its demonstrated ability to optimize complex flow control strategies. However, DRL algorithms often suffer from low sampling efficiency and require numerous interactions between the agent and the environment, necessitating frequent data exchanges. One significant bottleneck in coupled DRL–CFD algorithms is the extensive data communication between DRL and CFD codes. Non-intrusive algorithms where the DRL agent treats the CFD environment as a black box may come with the deficiency of increased computational cost due to overhead associated with the information exchange between the two DRL and CFD modules. In this article, a TensorFlow-based intrusive DRL–CFD framework is introduced where the agent model is integrated within the open-source CFD solver OpenFOAM. The integration eliminates the need for any external information exchange during DRL episodes. The framework is parallelized using the message passing interface to manage parallel environments for computationally intensive CFD cases through distributed computing. The performance and effectiveness of the framework are verified by controlling the vortex shedding behind two and three-dimensional cylinders, achieved as a result of minimizing drag and lift forces through an active flow control mechanism. The simulation results indicate that the trained controller can stabilize the flow and effectively mitigate the vortex shedding.

Intrusive

Active flow control

OpenFOAM

Deep reinforcement learning

Computational fluid dynamics

Author

Saeed Salehi

Chalmers, Mechanics and Maritime Sciences (M2), Fluid Dynamics

Meccanica

0025-6455 (ISSN) 1572-9648 (eISSN)

Vol. In Press

Artificial intelligence for enhanced hydraulic turbine lifetime

Energiforsk AB (VKU33020), 2023-01-01 -- 2027-06-30.

Swedish Energy Agency (VKU33020), 2023-01-01 -- 2027-06-30.

Areas of Advance

Energy

Infrastructure

C3SE (Chalmers Centre for Computational Science and Engineering)

Subject Categories

Fluid Mechanics and Acoustics

Computer Science

DOI

10.1007/s11012-024-01830-1

More information

Latest update

6/20/2024