Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution

Torgander, Jakob ; Magnusson, Måns and Wallin, Jonas LU (2024) Sixth Symposium on Advances in Approximate Bayesian Inference
Abstract
We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the "Gumbel Max Trick" and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Contribution to conference
publication status
published
subject
keywords
Hamiltonian Monte Carlo, MCMC, robabilistic machine learning, Gumbel max trick
pages
12 pages
conference name
Sixth Symposium on Advances in Approximate Bayesian Inference
conference location
Vienna, Austria
conference dates
2024-07-21 - 2024-07-21
language
English
LU publication?
yes
id
57bbfa82-3a66-4442-b8e8-c14f6f5b63cf
alternative location
https://openreview.net/forum?id=vyIUhU5LUr
date added to LUP
2025-03-05 10:00:19
date last changed
2025-06-23 10:39:42
@misc{57bbfa82-3a66-4442-b8e8-c14f6f5b63cf,
  abstract     = {{We introduce a method to enable Hamiltonian Monte Carlo (HMC) to simulate from mixed continuous and discrete posterior distributions. In particular, we show how the "Gumbel Max Trick" and the Concrete (Gumbel-softmax) distribution can be used for constructing a continuous approximation of a categorical distribution, and how this distribution can be efficiently implemented for HMC. We also illustrate how the Concrete distribution can be incorporated into a latent discrete parameter model, resulting in the Concrete Mixture model.}},
  author       = {{Torgander, Jakob and Magnusson, Måns and Wallin, Jonas}},
  keywords     = {{Hamiltonian Monte Carlo; MCMC; robabilistic machine learning; Gumbel max trick}},
  language     = {{eng}},
  month        = {{07}},
  title        = {{Hamiltonian Monte Carlo with categorical parameters using the Concrete distribution}},
  url          = {{https://openreview.net/forum?id=vyIUhU5LUr}},
  year         = {{2024}},
}