Advanced

On Bounds and Asymptotics of Sequential Monte Carlo Methods for Filtering, Smoothing, and Maximum Likelihood Estimation in State Space Models

Olsson, Jimmy LU (2007)
Abstract
This thesis is based on four papers (A-D) treating filtering, smoothing, and maximum likelihood (ML) estimation in general state space models using stochastic particle filters (also referred to as sequential Monte Carlo (SMC) methods).



The aim of Paper A is to study the bias of Monte Carlo integration estimates produced by the so-called bootstrap particle filter. A bound on this bias which is inversely proportional to the number N of particles of the system is established. In addition, we refine the analysis by deriving the asymptotic bias as N tends to infinity and, under suitable mixing assumptions on the latent Markov model, a time uniform bound.



In Paper B we consider ML estimation based on EM... (More)
This thesis is based on four papers (A-D) treating filtering, smoothing, and maximum likelihood (ML) estimation in general state space models using stochastic particle filters (also referred to as sequential Monte Carlo (SMC) methods).



The aim of Paper A is to study the bias of Monte Carlo integration estimates produced by the so-called bootstrap particle filter. A bound on this bias which is inversely proportional to the number N of particles of the system is established. In addition, we refine the analysis by deriving the asymptotic bias as N tends to infinity and, under suitable mixing assumptions on the latent Markov model, a time uniform bound.



In Paper B we consider ML estimation based on EM (Expectation-Maximization) methods. In this context, the key ingredient is the computation of smoothed sum functionals of the hidden states for given values of the model parameters. It has been observed by several authors that using standard SMC methods for this smoothing assignment may be unreliable for larger observations sizes. Thus we study a simple variant, based on forgetting ideas of the state space model dynamics, of the basic sequential smoothing approach which is transparent in terms of computation time and reduces the variability of the sum functional approximation. Under suitable regularity assumptions, it is shown that this modification indeed allows a tighter control of the L_p error and the bias of the approximation.



To perform ML estimation in state space models, the log-likelihood function must be approximated. In Paper C we study such approximations based on particle filters, and in particular conditions for consistency and asymptotic normality of the corresponding approximate ML estimators. Numerical results illustrate the theory.



Paper D is devoted to the study asymptotic properties of weighted particle samples produced by the so called two-stage sampling (TSS) particle filter, which is a generalization of the auxiliary particle filter proposed by Pitt and Shephard (1999). Besides establishing a central limit theorem (CLT) for smoothed particle estimates, we also derive bounds on the L_p error and bias of the same for a finite particle sample size. Setting out from the recursive formula for the asymptotic varianc of the CLT, we discuss some possible improvements of the TSS algorithm. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Professor Ritov, Ya'acov, The Hebrew University of Jerusalem, Jerusalem, Israel
organization
publishing date
type
Thesis
publication status
published
subject
keywords
actuarial mathematics, programming, operations research, Statistics, Matematik, Mathematics, state space models, smoothing, sequential Monte Carlo, particle filter, EM algorithm, maximum likelihood, consistency, Asymptotic normality, Statistik, operationsanalys, programmering, aktuariematematik
pages
182 pages
publisher
Lund University
defense location
Sal MH:A, Matematikcentrum, Sölvegatan 18, Lund
defense date
2007-01-26 10:15
external identifiers
  • other:ISRN: LUTFMS-1029-2006
ISSN
1404-0034
ISBN
91-628-7048-3
language
English
LU publication?
yes
id
cdb77001-c44c-47b3-9d28-117bb3fe93fb (old id 547827)
date added to LUP
2007-09-27 16:02:28
date last changed
2016-09-19 08:44:59
@phdthesis{cdb77001-c44c-47b3-9d28-117bb3fe93fb,
  abstract     = {This thesis is based on four papers (A-D) treating filtering, smoothing, and maximum likelihood (ML) estimation in general state space models using stochastic particle filters (also referred to as sequential Monte Carlo (SMC) methods).<br/><br>
<br/><br>
The aim of Paper A is to study the bias of Monte Carlo integration estimates produced by the so-called bootstrap particle filter. A bound on this bias which is inversely proportional to the number N of particles of the system is established. In addition, we refine the analysis by deriving the asymptotic bias as N tends to infinity and, under suitable mixing assumptions on the latent Markov model, a time uniform bound.<br/><br>
<br/><br>
In Paper B we consider ML estimation based on EM (Expectation-Maximization) methods. In this context, the key ingredient is the computation of smoothed sum functionals of the hidden states for given values of the model parameters. It has been observed by several authors that using standard SMC methods for this smoothing assignment may be unreliable for larger observations sizes. Thus we study a simple variant, based on forgetting ideas of the state space model dynamics, of the basic sequential smoothing approach which is transparent in terms of computation time and reduces the variability of the sum functional approximation. Under suitable regularity assumptions, it is shown that this modification indeed allows a tighter control of the L_p error and the bias of the approximation.<br/><br>
<br/><br>
To perform ML estimation in state space models, the log-likelihood function must be approximated. In Paper C we study such approximations based on particle filters, and in particular conditions for consistency and asymptotic normality of the corresponding approximate ML estimators. Numerical results illustrate the theory.<br/><br>
<br/><br>
Paper D is devoted to the study asymptotic properties of weighted particle samples produced by the so called two-stage sampling (TSS) particle filter, which is a generalization of the auxiliary particle filter proposed by Pitt and Shephard (1999). Besides establishing a central limit theorem (CLT) for smoothed particle estimates, we also derive bounds on the L_p error and bias of the same for a finite particle sample size. Setting out from the recursive formula for the asymptotic varianc of the CLT, we discuss some possible improvements of the TSS algorithm.},
  author       = {Olsson, Jimmy},
  isbn         = {91-628-7048-3},
  issn         = {1404-0034},
  keyword      = {actuarial mathematics,programming,operations research,Statistics,Matematik,Mathematics,state space models,smoothing,sequential Monte Carlo,particle filter,EM algorithm,maximum likelihood,consistency,Asymptotic normality,Statistik,operationsanalys,programmering,aktuariematematik},
  language     = {eng},
  pages        = {182},
  publisher    = {Lund University},
  school       = {Lund University},
  title        = {On Bounds and Asymptotics of Sequential Monte Carlo Methods for Filtering, Smoothing, and Maximum Likelihood Estimation in State Space Models},
  year         = {2007},
}