Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Adaptive Control of Positive Systems with Application to Learning SSP

Bencherki, Fethi LU and Rantzer, Anders LU orcid (2025) p.660-672
Abstract
An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal... (More)
An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment. (Less)
Please use this url to cite or link to this publication:
author
and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
7th Annual Learning for Dynamics and Control Conference
pages
13 pages
project
WASP NEST: Learning in Networks: Structure, Dynamics, and Control
language
English
LU publication?
yes
id
5ac30922-d637-42dc-bedf-871a9bec557c
alternative location
https://proceedings.mlr.press/v283/bencherki25a.html
date added to LUP
2025-07-27 17:02:20
date last changed
2025-09-03 10:01:41
@inproceedings{5ac30922-d637-42dc-bedf-871a9bec557c,
  abstract     = {{An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a “data-driven algebraic equation” constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment.}},
  author       = {{Bencherki, Fethi and Rantzer, Anders}},
  booktitle    = {{7th Annual Learning for Dynamics and Control Conference}},
  language     = {{eng}},
  pages        = {{660--672}},
  title        = {{Adaptive Control of Positive Systems with Application to Learning SSP}},
  url          = {{https://proceedings.mlr.press/v283/bencherki25a.html}},
  year         = {{2025}},
}