Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

On Exact Solutions to the Linear Bellman Equation

Ohlin, David LU orcid ; Pates, Richard LU and Arcak, Murat (2025) In IEEE Control Systems Letters 9. p.1568-1573
Abstract

This letter presents sufficient conditions for optimal control of systems with dynamics given by a linear operator, in order to obtain an explicit solution to the Bellman equation that can be calculated in a distributed fashion. Further, the class of Linearly Solvable MDP is reformulated as a continuous-state optimal control problem. It is shown that this class naturally satisfies the conditions for explicit solution of the Bellman equation, motivating the extension of previous results to semilinear dynamics to account for input nonlinearities. The applicability of the given conditions is illustrated in scenarios with linear and quadratic cost, corresponding to the Stochastic Shortest Path and Linear-Quadratic Regulator problems.

Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
keywords
compartmental and positive systems, linear systems, Markov processes, Optimal control
in
IEEE Control Systems Letters
volume
9
pages
6 pages
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
external identifiers
  • scopus:105009657645
ISSN
2475-1456
DOI
10.1109/LCSYS.2025.3582908
language
English
LU publication?
yes
additional info
Publisher Copyright: © 2017 IEEE.
id
0999b2af-f8db-479c-921f-dc8e64931300
date added to LUP
2026-01-13 13:08:01
date last changed
2026-01-13 13:09:10
@article{0999b2af-f8db-479c-921f-dc8e64931300,
  abstract     = {{<p>This letter presents sufficient conditions for optimal control of systems with dynamics given by a linear operator, in order to obtain an explicit solution to the Bellman equation that can be calculated in a distributed fashion. Further, the class of Linearly Solvable MDP is reformulated as a continuous-state optimal control problem. It is shown that this class naturally satisfies the conditions for explicit solution of the Bellman equation, motivating the extension of previous results to semilinear dynamics to account for input nonlinearities. The applicability of the given conditions is illustrated in scenarios with linear and quadratic cost, corresponding to the Stochastic Shortest Path and Linear-Quadratic Regulator problems.</p>}},
  author       = {{Ohlin, David and Pates, Richard and Arcak, Murat}},
  issn         = {{2475-1456}},
  keywords     = {{compartmental and positive systems; linear systems; Markov processes; Optimal control}},
  language     = {{eng}},
  pages        = {{1568--1573}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  series       = {{IEEE Control Systems Letters}},
  title        = {{On Exact Solutions to the Linear Bellman Equation}},
  url          = {{http://dx.doi.org/10.1109/LCSYS.2025.3582908}},
  doi          = {{10.1109/LCSYS.2025.3582908}},
  volume       = {{9}},
  year         = {{2025}},
}