Advanced

Approximate Dynamic Programming with Applications

Wernrud, Andreas LU (2008) In PhD Theses TFRT-1082.
Abstract
This thesis studies approximate optimal control of nonlinear systems.

Particular attention is given to global solutions and to the computation

of approximately optimal feedback controllers. The solution to an optimal

control problem is characterized by the optimal value function. For a

large class of problems the optimal value function must satisfy a

Hamilton-Jacobi-Bellman type equation. Two common methods for solving

such equations are policy iteration and value iteration. Both these

methods are studied in this thesis.



An approximate policy iteration algorithm is presented

for both the continuous and discrete time settings. It is shown that... (More)
This thesis studies approximate optimal control of nonlinear systems.

Particular attention is given to global solutions and to the computation

of approximately optimal feedback controllers. The solution to an optimal

control problem is characterized by the optimal value function. For a

large class of problems the optimal value function must satisfy a

Hamilton-Jacobi-Bellman type equation. Two common methods for solving

such equations are policy iteration and value iteration. Both these

methods are studied in this thesis.



An approximate policy iteration algorithm is presented

for both the continuous and discrete time settings. It is shown that

the sequence produced by this algorithm converges monotonically

towards the optimal value function. A multivariate polynomial

relaxation algorithm is proposed for linearly constrained discrete

time optimal control problems with convex cost. Relaxed value

iteration is studied for constrained linear systems with

convex piecewise linear cost. It is shown how an explicit

piecewise linear control law can be computed and how the resulting

lookup table can be reduced efficiently.



The on-line implementation of receding horizon controllers, even for

linear systems, is usually restricted to systems with slow dynamics.

One reason for this is that the delay between measurement and actuation

introduced by computing the control signal on-line can severely degrade

systems with fast dynamics. A method to improve robustness against

such delays and other uncertainties is presented.



A case study on the control of DC--DC converters is given. Feasibility

of a Relaxed Dynamic Programming algorithm is verified by synthesizing

controllers for both a step-down converter and a step-up converter.

The control performance is evaluated both in simulations and in real

experiments. (Less)
Please use this url to cite or link to this publication:
author
supervisor
opponent
  • Professor Parrilo, Pablo, Massachusetts Institute of Technology (MIT)
organization
publishing date
type
Thesis
publication status
published
subject
in
PhD Theses
volume
TFRT-1082
publisher
Department of Automatic Control, Lund Institute of Technology, Lund University
defense location
Room M:A, the M-building, Ole Römers väg 1, Lund University Faculty of Engineering
defense date
2008-02-29 10:15
ISSN
0280-5316
language
English
LU publication?
yes
id
bd85f9a7-02b8-43ad-95c6-f3b1962a296d (old id 1023673)
date added to LUP
2008-02-05 10:24:48
date last changed
2016-09-19 08:44:49
@phdthesis{bd85f9a7-02b8-43ad-95c6-f3b1962a296d,
  abstract     = {This thesis studies approximate optimal control of nonlinear systems. <br/><br>
Particular attention is given to global solutions and to the computation <br/><br>
of approximately optimal feedback controllers. The solution to an optimal <br/><br>
control problem is characterized by the optimal value function. For a <br/><br>
large class of problems the optimal value function must satisfy a <br/><br>
Hamilton-Jacobi-Bellman type equation. Two common methods for solving<br/><br>
such equations are policy iteration and value iteration. Both these <br/><br>
methods are studied in this thesis.<br/><br>
<br/><br>
An approximate policy iteration algorithm is presented <br/><br>
for both the continuous and discrete time settings. It is shown that <br/><br>
the sequence produced by this algorithm converges monotonically<br/><br>
towards the optimal value function. A multivariate polynomial<br/><br>
relaxation algorithm is proposed for linearly constrained discrete <br/><br>
time optimal control problems with convex cost. Relaxed value <br/><br>
iteration is studied for constrained linear systems with <br/><br>
convex piecewise linear cost. It is shown how an explicit <br/><br>
piecewise linear control law can be computed and how the resulting <br/><br>
lookup table can be reduced efficiently.<br/><br>
<br/><br>
The on-line implementation of receding horizon controllers, even for<br/><br>
linear systems, is usually restricted to systems with slow dynamics.<br/><br>
One reason for this is that the delay between measurement and actuation <br/><br>
introduced by computing the control signal on-line can severely degrade<br/><br>
systems with fast dynamics. A method to improve robustness against<br/><br>
such delays and other uncertainties is presented. <br/><br>
<br/><br>
A case study on the control of DC--DC converters is given. Feasibility <br/><br>
of a Relaxed Dynamic Programming algorithm is verified by synthesizing <br/><br>
controllers for both a step-down converter and a step-up converter.<br/><br>
The control performance is evaluated both in simulations and in real <br/><br>
experiments.},
  author       = {Wernrud, Andreas},
  issn         = {0280-5316},
  language     = {eng},
  publisher    = {Department of Automatic Control, Lund Institute of Technology, Lund University},
  school       = {Lund University},
  series       = {PhD Theses},
  title        = {Approximate Dynamic Programming with Applications},
  volume       = {TFRT-1082},
  year         = {2008},
}