Using Approximative Methods in Riemann Manifold Metropolis Adjusted Langevin Algorithm
(2012) FMS820 20122Mathematical Statistics
 Abstract (Swedish)
 When sampling from target densities with high correlation using the MetropolisHastings algorithm, performance can be improved by considering a discretization of a Langevin diffusion defined on the Riemann manifold as proposal mechanism. Through the metric tensor of the manifold, it is possible to capture the curvature of the parameter space and adjust for the correlation and differences in magnitude of variance. However, these methods need analytical expressions of several quantities, such as the expected Fisher information matrix and its derivative, which may be cumbersome to derive in more challenging problems. This calls for approximative methods.
In this bachelor thesis, two approximative methods are investigated. The first method... (More)  When sampling from target densities with high correlation using the MetropolisHastings algorithm, performance can be improved by considering a discretization of a Langevin diffusion defined on the Riemann manifold as proposal mechanism. Through the metric tensor of the manifold, it is possible to capture the curvature of the parameter space and adjust for the correlation and differences in magnitude of variance. However, these methods need analytical expressions of several quantities, such as the expected Fisher information matrix and its derivative, which may be cumbersome to derive in more challenging problems. This calls for approximative methods.
In this bachelor thesis, two approximative methods are investigated. The first method uses secondorder finite differences to approximate the metric tensor and the second method uses the gradient of the log posterior to approximate the expected Fisher information matrix and through this the metric tensor. The methods are assessed by performing inference on normal distributed and gamma distributed data and the results are compared to the results of an analytical implementation, a Metropolis adjusted Langevin algorithm not using the Riemann manifold and a Random Walk MetropolisHastings algorithm.
The approximative methods turn out to be computationally slower than the analytical implementation, but the time normalized effective sampling size is of the same magnitude as for the analytical method, and the same improvement compared to the Metropolis adjusted Langevin algorithm and the Random Walk MetropolisHastings algorithm is observed in both the analytical and the approximative methods. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/studentpapers/record/3166631
 author
 Erntell, Filip
 supervisor

 Johan LindstrÃ¶m ^{LU}
 organization
 course
 FMS820 20122
 year
 2012
 type
 H1  Master's Degree (One Year)
 subject
 language
 English
 id
 3166631
 date added to LUP
 20121112 09:59:23
 date last changed
 20121112 09:59:23
@misc{3166631, abstract = {When sampling from target densities with high correlation using the MetropolisHastings algorithm, performance can be improved by considering a discretization of a Langevin diffusion defined on the Riemann manifold as proposal mechanism. Through the metric tensor of the manifold, it is possible to capture the curvature of the parameter space and adjust for the correlation and differences in magnitude of variance. However, these methods need analytical expressions of several quantities, such as the expected Fisher information matrix and its derivative, which may be cumbersome to derive in more challenging problems. This calls for approximative methods. In this bachelor thesis, two approximative methods are investigated. The first method uses secondorder finite differences to approximate the metric tensor and the second method uses the gradient of the log posterior to approximate the expected Fisher information matrix and through this the metric tensor. The methods are assessed by performing inference on normal distributed and gamma distributed data and the results are compared to the results of an analytical implementation, a Metropolis adjusted Langevin algorithm not using the Riemann manifold and a Random Walk MetropolisHastings algorithm. The approximative methods turn out to be computationally slower than the analytical implementation, but the time normalized effective sampling size is of the same magnitude as for the analytical method, and the same improvement compared to the Metropolis adjusted Langevin algorithm and the Random Walk MetropolisHastings algorithm is observed in both the analytical and the approximative methods.}, author = {Erntell, Filip}, language = {eng}, note = {Student Paper}, title = {Using Approximative Methods in Riemann Manifold Metropolis Adjusted Langevin Algorithm}, year = {2012}, }