Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Path Planning Using Wasserstein Distributionally Robust Deep Q-learning

Alpturk, Cem and Renganathan, Venkatraman LU (2023) 2023 European Control Conference, ECC 2023
Abstract

We investigate the problem of risk averse robot path planning using the deep reinforcement learning and distributionally robust optimization perspectives. Our problem formulation involves modelling the robot as a stochastic linear dynamical system, assuming that a collection of process noise samples is available. We cast the risk averse motion planning problem as a Markov decision process and propose a continuous reward function design that explicitly takes into account the risk of collision with obstacles while encouraging the robot's motion towards the goal. We learn the risk-averse robot control actions through Lipschitz approximated Wasserstein distributionally robust deep Q-learning to hedge against the noise uncertainty. The... (More)

We investigate the problem of risk averse robot path planning using the deep reinforcement learning and distributionally robust optimization perspectives. Our problem formulation involves modelling the robot as a stochastic linear dynamical system, assuming that a collection of process noise samples is available. We cast the risk averse motion planning problem as a Markov decision process and propose a continuous reward function design that explicitly takes into account the risk of collision with obstacles while encouraging the robot's motion towards the goal. We learn the risk-averse robot control actions through Lipschitz approximated Wasserstein distributionally robust deep Q-learning to hedge against the noise uncertainty. The learned control actions result in a safe and risk averse trajectory from the source to the goal, avoiding all the obstacles. Various supporting numerical simulations are presented to demonstrate our proposed approach.

(Less)
Please use this url to cite or link to this publication:
author
and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
2023 European Control Conference, ECC 2023
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
2023 European Control Conference, ECC 2023
conference location
Bucharest, Romania
conference dates
2023-06-13 - 2023-06-16
external identifiers
  • scopus:85166467960
ISBN
9783907144084
DOI
10.23919/ECC57647.2023.10178154
language
English
LU publication?
yes
id
0008f741-f793-4c20-be16-5907e4f84abc
date added to LUP
2023-11-21 11:40:29
date last changed
2023-11-21 11:41:48
@inproceedings{0008f741-f793-4c20-be16-5907e4f84abc,
  abstract     = {{<p>We investigate the problem of risk averse robot path planning using the deep reinforcement learning and distributionally robust optimization perspectives. Our problem formulation involves modelling the robot as a stochastic linear dynamical system, assuming that a collection of process noise samples is available. We cast the risk averse motion planning problem as a Markov decision process and propose a continuous reward function design that explicitly takes into account the risk of collision with obstacles while encouraging the robot's motion towards the goal. We learn the risk-averse robot control actions through Lipschitz approximated Wasserstein distributionally robust deep Q-learning to hedge against the noise uncertainty. The learned control actions result in a safe and risk averse trajectory from the source to the goal, avoiding all the obstacles. Various supporting numerical simulations are presented to demonstrate our proposed approach.</p>}},
  author       = {{Alpturk, Cem and Renganathan, Venkatraman}},
  booktitle    = {{2023 European Control Conference, ECC 2023}},
  isbn         = {{9783907144084}},
  language     = {{eng}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{Path Planning Using Wasserstein Distributionally Robust Deep Q-learning}},
  url          = {{http://dx.doi.org/10.23919/ECC57647.2023.10178154}},
  doi          = {{10.23919/ECC57647.2023.10178154}},
  year         = {{2023}},
}