Real-world deployment of model-free reinforcement learning for energy control in district heating systems : Enhancing flexibility across neighboring buildings
Moshari, Amirhosein LU ; Javanroodi, Kavan LU and Nik, Vahid M. LU
(2026)
In Applied Energy
402.
- Abstract
Energy Management Systems (EMSs) often operate through inflexible, rule-based control systems. Model-free reinforcement learning (RL) has emerged as a promising alternative, providing adaptive and autonomous control without the need for detailed modeling. However, the complexity of operating energy systems and dynamic environmental conditions limits their practical use, with most studies relying on simulations or short-term trials that fall short of real-world deployment. This research presents a prolonged, multi-building implementation of a complete, autonomous, model-free RL system that is developed and deployed within operational buildings throughout a full heating season. The system operated successfully without the need for... (More)
Energy Management Systems (EMSs) often operate through inflexible, rule-based control systems. Model-free reinforcement learning (RL) has emerged as a promising alternative, providing adaptive and autonomous control without the need for detailed modeling. However, the complexity of operating energy systems and dynamic environmental conditions limits their practical use, with most studies relying on simulations or short-term trials that fall short of real-world deployment. This research presents a prolonged, multi-building implementation of a complete, autonomous, model-free RL system that is developed and deployed within operational buildings throughout a full heating season. The system operated successfully without the need for simulation, pre-training, calibration, or additional sensor requirements, and, via flexibility signals, it facilitated a more efficient and robust deployment, while also addressing data privacy concerns. The analysis provided significant results, including a 7.9% decrease from the previous warmer year and a 29.7% reduction in heating energy compared to a multi-year historical baseline. The optimal performance of the implemented system became evident through a 3.85 °C reduction in return temperature and consistent reductions in peak demand, while maintaining occupant comfort. By combining ANCOVA normalization, matched-temperature baselines, thermodynamic efficiency metrics, and comfort analyses, this study introduces a structured framework and evaluation metrics to support robust field assessments of RL-based control in buildings. Finally, the study highlights the strong potential of autonomous RL engines for broader implementation in complex real-world energy management systems, as they require no knowledge of system dynamics or infrastructure upgrades, offering a reliable and cost-effective solution.
(Less)
- author
- Moshari, Amirhosein
LU
; Javanroodi, Kavan
LU
and Nik, Vahid M.
LU
- organization
- publishing date
- 2026-01-01
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- Autonomous building control, Building energy management, District heating networks, Energy efficiency, Real-world deployment, Reinforcement learning (RL)
- in
- Applied Energy
- volume
- 402
- article number
- 126997
- publisher
- Elsevier
- external identifiers
-
- scopus:105020937658
- ISSN
- 0306-2619
- DOI
- 10.1016/j.apenergy.2025.126997
- language
- English
- LU publication?
- yes
- additional info
- Publisher Copyright: © 2025 The Authors
- id
- 1c288309-ca60-412f-b2dd-eaae5759af79
- date added to LUP
- 2026-02-03 13:53:43
- date last changed
- 2026-02-17 10:06:27
@article{1c288309-ca60-412f-b2dd-eaae5759af79,
abstract = {{<p>Energy Management Systems (EMSs) often operate through inflexible, rule-based control systems. Model-free reinforcement learning (RL) has emerged as a promising alternative, providing adaptive and autonomous control without the need for detailed modeling. However, the complexity of operating energy systems and dynamic environmental conditions limits their practical use, with most studies relying on simulations or short-term trials that fall short of real-world deployment. This research presents a prolonged, multi-building implementation of a complete, autonomous, model-free RL system that is developed and deployed within operational buildings throughout a full heating season. The system operated successfully without the need for simulation, pre-training, calibration, or additional sensor requirements, and, via flexibility signals, it facilitated a more efficient and robust deployment, while also addressing data privacy concerns. The analysis provided significant results, including a 7.9% decrease from the previous warmer year and a 29.7% reduction in heating energy compared to a multi-year historical baseline. The optimal performance of the implemented system became evident through a 3.85 °C reduction in return temperature and consistent reductions in peak demand, while maintaining occupant comfort. By combining ANCOVA normalization, matched-temperature baselines, thermodynamic efficiency metrics, and comfort analyses, this study introduces a structured framework and evaluation metrics to support robust field assessments of RL-based control in buildings. Finally, the study highlights the strong potential of autonomous RL engines for broader implementation in complex real-world energy management systems, as they require no knowledge of system dynamics or infrastructure upgrades, offering a reliable and cost-effective solution.</p>}},
author = {{Moshari, Amirhosein and Javanroodi, Kavan and Nik, Vahid M.}},
issn = {{0306-2619}},
keywords = {{Autonomous building control; Building energy management; District heating networks; Energy efficiency; Real-world deployment; Reinforcement learning (RL)}},
language = {{eng}},
month = {{01}},
publisher = {{Elsevier}},
series = {{Applied Energy}},
title = {{Real-world deployment of model-free reinforcement learning for energy control in district heating systems : Enhancing flexibility across neighboring buildings}},
url = {{http://dx.doi.org/10.1016/j.apenergy.2025.126997}},
doi = {{10.1016/j.apenergy.2025.126997}},
volume = {{402}},
year = {{2026}},
}