Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Approximate Dynamic Programming with Applications

Wernrud, Andreas LU (2008) In PhD Thesis TFRT-1082
Abstract
This thesis studies approximate optimal control of nonlinear systems. Particular attention is given to global solutions and to the computation of approximately optimal feedback controllers. The solution to an optimal control problem is characterized by the optimal value function. For a large class of problems the optimal value function must satisfy a Hamilton-Jacobi-Bellman type equation. Two common methods for solving such equations are policy iteration and value iteration. Both these methods are studied in this thesis.

An approximate policy iteration algorithm is presented for both the continuous and discrete time settings. It is shown that the sequence produced by this algorithm converges monotonically towards the optimal value... (More)
This thesis studies approximate optimal control of nonlinear systems. Particular attention is given to global solutions and to the computation of approximately optimal feedback controllers. The solution to an optimal control problem is characterized by the optimal value function. For a large class of problems the optimal value function must satisfy a Hamilton-Jacobi-Bellman type equation. Two common methods for solving such equations are policy iteration and value iteration. Both these methods are studied in this thesis.

An approximate policy iteration algorithm is presented for both the continuous and discrete time settings. It is shown that the sequence produced by this algorithm converges monotonically towards the optimal value function. A multivariate polynomial relaxation algorithm is proposed for linearly constrained discrete time optimal control problems with convex cost. Relaxed value iteration is studied for constrained linear systems with convex piecewise linear cost. It is shown how an explicit piecewise linear control law can be computed and how the resulting lookup table can be reduced efficiently.

The on-line implementation of receding horizon controllers, even for linear systems, is usually restricted to systems with slow dynamics. One reason for this is that the delay between measurement and actuation introduced by computing the control signal on-line can severely degrade systems with fast dynamics. A method to improve robustness against such delays and other uncertainties is presented. A case study on the control of DC--DC converters is given. Feasibility of a Relaxed Dynamic Programming algorithm is verified by synthesizing controllers for both a step-down converter and a step-up converter. The control performance is evaluated both in simulations and in real experiments. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Professor Parrilo, Pablo, Massachusetts Institute of Technology (MIT)
organization
publishing date
type
Thesis
publication status
published
subject
in
PhD Thesis TFRT-1082
publisher
Department of Automatic Control, Lund Institute of Technology, Lund University
defense location
Room M:A, the M-building, Ole Römers väg 1, Lund University Faculty of Engineering
defense date
2008-02-29 10:15:00
ISSN
0280-5316
0280-5316
language
English
LU publication?
yes
id
bd85f9a7-02b8-43ad-95c6-f3b1962a296d (old id 1023673)
date added to LUP
2016-04-01 14:18:20
date last changed
2019-05-23 15:59:51
@phdthesis{bd85f9a7-02b8-43ad-95c6-f3b1962a296d,
  abstract     = {{This thesis studies approximate optimal control of nonlinear systems. Particular attention is given to global solutions and to the computation of approximately optimal feedback controllers. The solution to an optimal control problem is characterized by the optimal value function. For a large class of problems the optimal value function must satisfy a Hamilton-Jacobi-Bellman type equation. Two common methods for solving such equations are policy iteration and value iteration. Both these methods are studied in this thesis.<br/><br/>An approximate policy iteration algorithm is presented for both the continuous and discrete time settings. It is shown that the sequence produced by this algorithm converges monotonically towards the optimal value function. A multivariate polynomial relaxation algorithm is proposed for linearly constrained discrete time optimal control problems with convex cost. Relaxed value iteration is studied for constrained linear systems with convex piecewise linear cost. It is shown how an explicit piecewise linear control law can be computed and how the resulting lookup table can be reduced efficiently.<br/><br/>The on-line implementation of receding horizon controllers, even for linear systems, is usually restricted to systems with slow dynamics. One reason for this is that the delay between measurement and actuation introduced by computing the control signal on-line can severely degrade systems with fast dynamics. A method to improve robustness against such delays and other uncertainties is presented.  A case study on the control of DC--DC converters is given. Feasibility of a Relaxed Dynamic Programming algorithm is verified by synthesizing controllers for both a step-down converter and a step-up converter. The control performance is evaluated both in simulations and in real experiments.}},
  author       = {{Wernrud, Andreas}},
  issn         = {{0280-5316}},
  language     = {{eng}},
  publisher    = {{Department of Automatic Control, Lund Institute of Technology, Lund University}},
  school       = {{Lund University}},
  series       = {{PhD Thesis TFRT-1082}},
  title        = {{Approximate Dynamic Programming with Applications}},
  url          = {{https://lup.lub.lu.se/search/files/3898504/1023689.pdf}},
  year         = {{2008}},
}