Reinforcement Learning — From Simulation to Deployment on the Robot Platform Spot
(2026)Department of Automatic Control
- Abstract
- Recent advancements in GPU-accelerated simulation have made it possible to train reinforcement learning policies for robotic control using large-scale parallel data collection. This thesis explores the pipeline from task description to deployment of control policies on the robot platform Spot. The focus is on two important skills needed for future loco-manipulation tasks: robust standing and command-following locomotion. The thesis presents the design choices made when creating an Isaac Lab environment, including the observation space, action space, reward functions, and deployment setup. The resulting policies were tested both in simulation and on real Spot hardware using the Spot RL Researcher Kit. The experiments showed that the trained... (More)
- Recent advancements in GPU-accelerated simulation have made it possible to train reinforcement learning policies for robotic control using large-scale parallel data collection. This thesis explores the pipeline from task description to deployment of control policies on the robot platform Spot. The focus is on two important skills needed for future loco-manipulation tasks: robust standing and command-following locomotion. The thesis presents the design choices made when creating an Isaac Lab environment, including the observation space, action space, reward functions, and deployment setup. The resulting policies were tested both in simulation and on real Spot hardware using the Spot RL Researcher Kit. The experiments showed that the trained policies could be deployed on hardware and that successful locomotion was possible. At the same time, a sim-to-real gap was observed, especially in the joint tracking and load-bearing behaviour of the robot. The thesis therefore shows both the potential of the pipeline and the practical challenges involved in transferring reinforcement learning policies from simulation to real hardware deployment. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/student-papers/record/9251353
- author
- Lejon, Markus and Sundt, Fredrik
- supervisor
-
- Volker Krueger LU
- Björn Olofsson LU
- Karl-Erik Årzén LU
- organization
- year
- 2026
- type
- H3 - Professional qualifications (4 Years - )
- subject
- report number
- TFRT-6326
- other publication id
- 0280-5316
- language
- English
- id
- 9251353
- date added to LUP
- 2026-09-18 11:47:01
- date last changed
- 2026-09-18 11:47:01
@misc{9251353,
abstract = {{Recent advancements in GPU-accelerated simulation have made it possible to train reinforcement learning policies for robotic control using large-scale parallel data collection. This thesis explores the pipeline from task description to deployment of control policies on the robot platform Spot. The focus is on two important skills needed for future loco-manipulation tasks: robust standing and command-following locomotion. The thesis presents the design choices made when creating an Isaac Lab environment, including the observation space, action space, reward functions, and deployment setup. The resulting policies were tested both in simulation and on real Spot hardware using the Spot RL Researcher Kit. The experiments showed that the trained policies could be deployed on hardware and that successful locomotion was possible. At the same time, a sim-to-real gap was observed, especially in the joint tracking and load-bearing behaviour of the robot. The thesis therefore shows both the potential of the pipeline and the practical challenges involved in transferring reinforcement learning policies from simulation to real hardware deployment.}},
author = {{Lejon, Markus and Sundt, Fredrik}},
language = {{eng}},
note = {{Student Paper}},
title = {{Reinforcement Learning — From Simulation to Deployment on the Robot Platform Spot}},
year = {{2026}},
}