Inference techniques for stochastic nonlinear system identification with application to the Wiener-Hammerstein models
Doktorsavhandling, 2018

Stochastic nonlinear systems are a specific class of nonlinear systems where unknown disturbances affect the system's output through a nonlinear transformation. In general, the identification of parametric models for this kind of systems can be very challenging. A main statistical inference technique for parameter estimation is the Maximum Likelihood estimator. The central object of this technique is the likelihood function, i.e. a mathematical expression describing the probability of obtaining certain observations for given values of the parameter. For many stochastic nonlinear systems, however, the likelihood function is not available in closed-form. Several methods have been developed to obtain approximate solutions to the Maximum Likelihood problem, mainly based on the Monte Carlo method. However, one of the main difficulties of these methods is that they can be computationally expensive, especially when they are combined with numerical optimization techniques for likelihood maximisation.

This thesis can be divided in three parts. In the first part, a background on the main statistical techniques for parameter estimation is presented. In particular, two iterative methods for finding the Maximum Likelihood estimator are introduced. They are the gradient-based and the Expectation-Maximisation algorithms.

In the second part, the main Monte Carlo methods for approximating the Maximum Likelihood problem are analysed. Their combinations with gradient-based and Expectation-Maximisation algorithms is considered. For ensuring convergence, these algorithms require the use of enormous Monte Carlo effort, i.e. the number of random samples used to build the Monte Carlo estimates. In order to reduce this effort and make the algorithms usable in practice, iterative solutions solutions alternating \emph{local} Monte Carlo approximations and maximisation steps are derived. In particular, a procedure implementing an efficient samples simulation across the steps of a Newton's method is developed. The procedure is based on the sensitivity of the parameter search with respect to the Monte Carlo samples and it results into an accurate and fast algorithm for solving the MLE problem.

The considered Maximum Likelihood estimation methods proceed through local explorations of the parameter space. Hence, they have guaranteed convergence only to a local optimizer of the likelihood function. In the third part of the thesis, this issue is addressed by deriving initialization algorithms. The purpose is to generate initial guesses that increase the chances of converging to the global maximum. In particular, initialization algorithms are derived for the Wiener-Hammerstein model, i.e. a nonlinear model where a static nonlinearity is sandwiched between two linear parts. For this type of model, it can be proved that the best linear approximation of the system provides a consistent estimates of the two linear parts. This estimate is then used to initialize a Maximum Likelihood Estimation problem in all model parameters.

nonlinear systems

system identification


Maximum Likelihood


Monte Carlo

Newton's method

Department Electrical Engineering
Opponent: Professor Roy Smith, ETH Zurich, Switzerland


Giuseppe Giordano

Chalmers, Elektroteknik, System- och reglerteknik

Maximum Likelihood identification of Wiener-Hammerstein system with process noise

IFAC-PapersOnLine,; Vol. 51(2018)p. 401-406

Paper i proceeding

An improved method for Wiener–Hammerstein system identification based on the Fractional Approach

Automatica,; Vol. 94(2018)p. 349-360

Artikel i vetenskaplig tidskrift

Consistency aspects of Wiener-Hammerstein model identification in presence of process noise

2016 IEEE 55th Conference on Decision and Control, CDC 2016,; (2016)p. Art no 7798724, Pages 3042-3047

Paper i proceeding

A Time-Domain Fractional Approach for Wiener-Hammerstein Systems Identification

IFAC-PapersOnLine,; Vol. 48(2015)p. 1232-1237

Paper i proceeding

G. Giordano, S. Gros, J. Sjöberg, “A Newton-based method for Maximum Likelihood estimation from incomplete data”, to be submitted to Automatica, 2018.



Sannolikhetsteori och statistik




Doktorsavhandlingar vid Chalmers tekniska högskola. Ny serie: 4471



Department Electrical Engineering

Opponent: Professor Roy Smith, ETH Zurich, Switzerland

Mer information

Senast uppdaterat