A retrospective on the robot air hockey challenge : benchmarking robust, reliable, and safe learning techniques for real-world robotics
(2024) 38th Conference on Neural Information Processing Systems, NeurIPS 2024 In Advances in Neural Information Processing Systems 37.- Abstract
Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark,... (More)
Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Furthermore, we focus on a dynamic environment, removing the typical assumption of quasi-static motions of other real-world benchmarks. The competition's results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging. Our ablation study reveals which real-world factors may be overlooked when building a learning-based solution. The successful real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.
(Less)
- author
- Liu, Puze
; Günster, Jonas
; Funk, Niklas
; Gröger, Simon
; Chen, Dong
; Bou-Ammar, Haitham
; Jankowski, Julius
; Marić, Ante
; Calinon, Sylvain
; Orsula, Andrej
; Olivares-Mendez, Miguel
; Zhou, Hongyi
; Lioutikov, Rudolf
; Neumann, Gerhard
; Likmeta, Amarildo
; Zhalehmehrabi, Amirhossein
; Bonenfant, Thomas
; Restelli, Marcello
; Tateo, Davide
LU
; Liu, Ziyuan
and Peters, Jan
(Less) - publishing date
- 2024
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track
- series title
- Advances in Neural Information Processing Systems
- volume
- 37
- conference name
- 38th Conference on Neural Information Processing Systems, NeurIPS 2024
- conference location
- Vancouver, Canada
- conference dates
- 2024-12-09 - 2024-12-15
- external identifiers
-
- scopus:105000479399
- ISSN
- 1049-5258
- language
- English
- LU publication?
- no
- id
- 7743d828-f078-48df-a89f-a7b0297d04a0
- alternative location
- https://proceedings.neurips.cc/paper_files/paper/2024/hash/12ba5de27afcff1a5c796de4a6392154-Abstract-Datasets_and_Benchmarks_Track.html
- date added to LUP
- 2025-10-16 14:15:46
- date last changed
- 2025-11-29 03:46:22
@inproceedings{7743d828-f078-48df-a89f-a7b0297d04a0,
abstract = {{<p>Machine learning methods have a groundbreaking impact in many application domains, but their application on real robotic platforms is still limited. Despite the many challenges associated with combining machine learning technology with robotics, robot learning remains one of the most promising directions for enhancing the capabilities of robots. When deploying learning-based approaches on real robots, extra effort is required to address the challenges posed by various real-world factors. To investigate the key factors influencing real-world deployment and to encourage original solutions from different researchers, we organized the Robot Air Hockey Challenge at the NeurIPS 2023 conference. We selected the air hockey task as a benchmark, encompassing low-level robotics problems and high-level tactics. Different from other machine learning-centric benchmarks, participants need to tackle practical challenges in robotics, such as the sim-to-real gap, low-level control issues, safety problems, real-time requirements, and the limited availability of real-world data. Furthermore, we focus on a dynamic environment, removing the typical assumption of quasi-static motions of other real-world benchmarks. The competition's results show that solutions combining learning-based approaches with prior knowledge outperform those relying solely on data when real-world deployment is challenging. Our ablation study reveals which real-world factors may be overlooked when building a learning-based solution. The successful real-world air hockey deployment of best-performing agents sets the foundation for future competitions and follow-up research directions.</p>}},
author = {{Liu, Puze and Günster, Jonas and Funk, Niklas and Gröger, Simon and Chen, Dong and Bou-Ammar, Haitham and Jankowski, Julius and Marić, Ante and Calinon, Sylvain and Orsula, Andrej and Olivares-Mendez, Miguel and Zhou, Hongyi and Lioutikov, Rudolf and Neumann, Gerhard and Likmeta, Amarildo and Zhalehmehrabi, Amirhossein and Bonenfant, Thomas and Restelli, Marcello and Tateo, Davide and Liu, Ziyuan and Peters, Jan}},
booktitle = {{The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track}},
issn = {{1049-5258}},
language = {{eng}},
series = {{Advances in Neural Information Processing Systems}},
title = {{A retrospective on the robot air hockey challenge : benchmarking robust, reliable, and safe learning techniques for real-world robotics}},
url = {{https://proceedings.neurips.cc/paper_files/paper/2024/hash/12ba5de27afcff1a5c796de4a6392154-Abstract-Datasets_and_Benchmarks_Track.html}},
volume = {{37}},
year = {{2024}},
}