Semantic Synthesis of Pedestrian Locomotion
(2021) 15th Asian Conference on Computer Vision, ACCV 2020 In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 12623 LNCS. p.470-487- Abstract
We present a model for generating 3d articulated pedestrian locomotion in urban scenarios, with synthesis capabilities informed by the 3d scene semantics and geometry. We reformulate pedestrian trajectory forecasting as a structured reinforcement learning (RL) problem. This allows us to naturally combine prior knowledge on collision avoidance, 3d human motion capture and the motion of pedestrians as observed e.g. in Cityscapes, Waymo or simulation environments like Carla. Our proposed RL-based model allows pedestrians to accelerate and slow down to avoid imminent danger (e.g. cars), while obeying human dynamics learnt from in-lab motion capture datasets. Specifically, we propose a hierarchical model consisting of a semantic trajectory... (More)
We present a model for generating 3d articulated pedestrian locomotion in urban scenarios, with synthesis capabilities informed by the 3d scene semantics and geometry. We reformulate pedestrian trajectory forecasting as a structured reinforcement learning (RL) problem. This allows us to naturally combine prior knowledge on collision avoidance, 3d human motion capture and the motion of pedestrians as observed e.g. in Cityscapes, Waymo or simulation environments like Carla. Our proposed RL-based model allows pedestrians to accelerate and slow down to avoid imminent danger (e.g. cars), while obeying human dynamics learnt from in-lab motion capture datasets. Specifically, we propose a hierarchical model consisting of a semantic trajectory policy network that provides a distribution over possible movements, and a human locomotion network that generates 3d human poses in each step. The RL-formulation allows the model to learn even from states that are seldom exhibited in the dataset, utilizing all of the available prior and scene information. Extensive evaluations using both real and simulated data illustrate that the proposed model is on par with recent models such as S-GAN, ST-GAT and S-STGCNN in pedestrian forecasting, while outperforming these in collision avoidance. We also show that our model can be used to plan goal reaching trajectories in urban scenes with dynamic actors.
(Less)
- author
- Priisalu, Maria LU ; Paduraru, Ciprian ; Pirinen, Aleksis LU and Sminchisescu, Cristian LU
- organization
- publishing date
- 2021
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers
- series title
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- editor
- Ishikawa, Hiroshi ; Liu, Cheng-Lin ; Pajdla, Tomas and Shi, Jianbo
- volume
- 12623 LNCS
- pages
- 18 pages
- publisher
- Springer Science and Business Media B.V.
- conference name
- 15th Asian Conference on Computer Vision, ACCV 2020
- conference location
- Virtual, Online
- conference dates
- 2020-11-30 - 2020-12-04
- external identifiers
-
- scopus:85103292089
- ISSN
- 0302-9743
- 1611-3349
- ISBN
- 9783030695316
- DOI
- 10.1007/978-3-030-69532-3_29
- project
- Modelling Pedestrians in Autonomous Vehicle Testing
- language
- English
- LU publication?
- yes
- id
- 0d2faa60-34ef-4c43-a4f7-718a6f695001
- date added to LUP
- 2021-04-08 12:32:45
- date last changed
- 2025-04-04 15:04:45
@inproceedings{0d2faa60-34ef-4c43-a4f7-718a6f695001,
abstract = {{<p>We present a model for generating 3d articulated pedestrian locomotion in urban scenarios, with synthesis capabilities informed by the 3d scene semantics and geometry. We reformulate pedestrian trajectory forecasting as a structured reinforcement learning (RL) problem. This allows us to naturally combine prior knowledge on collision avoidance, 3d human motion capture and the motion of pedestrians as observed e.g. in Cityscapes, Waymo or simulation environments like Carla. Our proposed RL-based model allows pedestrians to accelerate and slow down to avoid imminent danger (e.g. cars), while obeying human dynamics learnt from in-lab motion capture datasets. Specifically, we propose a hierarchical model consisting of a semantic trajectory policy network that provides a distribution over possible movements, and a human locomotion network that generates 3d human poses in each step. The RL-formulation allows the model to learn even from states that are seldom exhibited in the dataset, utilizing all of the available prior and scene information. Extensive evaluations using both real and simulated data illustrate that the proposed model is on par with recent models such as S-GAN, ST-GAT and S-STGCNN in pedestrian forecasting, while outperforming these in collision avoidance. We also show that our model can be used to plan goal reaching trajectories in urban scenes with dynamic actors.</p>}},
author = {{Priisalu, Maria and Paduraru, Ciprian and Pirinen, Aleksis and Sminchisescu, Cristian}},
booktitle = {{Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers}},
editor = {{Ishikawa, Hiroshi and Liu, Cheng-Lin and Pajdla, Tomas and Shi, Jianbo}},
isbn = {{9783030695316}},
issn = {{0302-9743}},
language = {{eng}},
pages = {{470--487}},
publisher = {{Springer Science and Business Media B.V.}},
series = {{Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)}},
title = {{Semantic Synthesis of Pedestrian Locomotion}},
url = {{http://dx.doi.org/10.1007/978-3-030-69532-3_29}},
doi = {{10.1007/978-3-030-69532-3_29}},
volume = {{12623 LNCS}},
year = {{2021}},
}