Semi-analytical MAP Estimation of Hidden Dynamics in Continuous-State POMDPs

Licentiate thesis, 2026

This thesis explores a problem in dynamical systems control with a finite action set. A common situation is that the underlying dynamics of the system is unknown and needs to be estimated based on observed data. However, when only partial information of the latent state space is available, and the latent state space is continuous, this task becomes exceedingly difficult. Unlike probabilistic sampling-based methods for solving this problem, a semi-analytical sampling-free procedure is examined.
We propose a representation framework based on weighted sums of basis functions for approximating a transition probability density function and show how this can be used with a truncated Gaussian prior probability measure in a semi-analytical algorithm for calculating the posterior log-probability of a represented transition PDF. We also provide efficient forward-backward algorithms for first- and second-order differentiation of this posterior which can either be used weight-wise to calculate a gradient and Hessian over the weights or functionally to calculate a representation of the functional first- and second-derivative.
Furthermore, semi-analytical optimization approaches for finding a maximum a posteriori estimate are examined, including a first- and a second-order function optimization method. These methods use a pricing oracle approach with active set iteration to dynamically grow the representation. Finally, the computational complexity of the various steps as well as theoretical convergence properties are analysed.