Methods for Stochastic Optimal Control under State Constraints
Doctoral thesis, 2017

This thesis looks at a few different approaches to solving stochas-tic optimal control problems with state constraints. The motivatingproblem is optimal control of an energy buffer in a hybrid vehicle,although applications are abundant in a number of areas.Stochastic optimal control problems can be solved via the so-called Hamilton-Jacobi-Bellman (HJB) equation. State constraintsresult in boundary conditions for the HJB equation causing the valuefunction to go to infinity as the state approaches the boundary, whichmakes it difficult to solve this partial differential equation numerically.Different approaches to avoiding infinite values on the boundaryare investigated. First, we consider a logarithmic transformation ofthe value function. This results in an exact linearization, turningtheHJB equation into an eigenvalue problem in the one-dimensional case,and also in higher dimensions, but then with certain restrictions onthe relation between noise and control cost. Then, for a more generalproblem formulation, we introduce a different transform which yieldsa nonlinear problem. It is investigated under what conditions theboundary constraints will be well-behaved, and example problemsare solved using a collocation method, demonstrating how a smallnumber of collocation points is sufficient to yield good solutions inthose cases. Finally, we consider a method starting from the Fokker-Planck equation. This yields an equivalent problem, but where thevalue function of the HJB equation need not be computed explic-itly, but the probability density function of the closed-loop system iscomputed instead. This fact can be utilized to focus computationalresources on the parts of the state-space that are the most relevant.

control theory

optimal control

stochastic systems


Fokker-Planck-Kolmogorov equation

SB-H4, Sven Hultins gata 8
Opponent: Prof. Anders Rantzer, Lunds Universitet


Per Rutquist

Electrical Engineering

State constrained control based on linearization of the Hamilton-Jacobi-Bellman equation

Proceedings of 49th IEEE Conference on Decision and Control. Atlanta, 15-17 December 2010,; (2010)p. 5192-5197

Paper in proceeding

An eigenvalue approach to infinite-horizon optimal control

Proceedings of the 16 th IFAC World Congress, Prague, Czech Republik,; (2005)

Paper in proceeding

Solving the Hamilton-Jacobi-Bellman equation for a stochastic system with state constraints

Proceedings of the 53rd IEEE Annual Conference on Decision and Control, CDC 2014, Los Angeles, United States, 15-17 December 2014,; (2014)p. 1840-1845

Paper in proceeding

On the infinite-time solution to state-constrained stochastic optimal control

Automatica,; Vol. 44(2008)p. 1800-1805

Journal article

Finite-time state-constrained optimal control for input-affine systems with actuator noise

IFAC Proceedings Volumes (IFAC-PapersOnline),; Vol. 18(2011)p. 5915-5919

Paper in proceeding

Subject Categories

Computational Mathematics

Control Engineering

Mathematical Analysis



Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 4303



SB-H4, Sven Hultins gata 8

Opponent: Prof. Anders Rantzer, Lunds Universitet

More information


9/4/2017 1