Distributed Reinforcement Learning for Building Energy Optimization
(2024)Department of Automatic Control
- Abstract
- Buildings worldwide account for roughly 40% of the total energy consumption which boils down to generating 30% of the total carbon dioxide emissions. Heating, Ventilation, and Air Conditioning (HVAC) systems are the largest consumers of energy within these structures. The complexity of building cooling systems poses significant challenges for achieving optimal control. Conventional approaches to improve energy efficiency in buildings rely on high-performance control systems, which demand substantial time and expertise. Reinforcement Learning (RL) has emerged as a potentially transformative technology for the control and optimization of HVAC systems. This project focuses on the use of (distributed) reinforcement learning as a supervisory... (More)
- Buildings worldwide account for roughly 40% of the total energy consumption which boils down to generating 30% of the total carbon dioxide emissions. Heating, Ventilation, and Air Conditioning (HVAC) systems are the largest consumers of energy within these structures. The complexity of building cooling systems poses significant challenges for achieving optimal control. Conventional approaches to improve energy efficiency in buildings rely on high-performance control systems, which demand substantial time and expertise. Reinforcement Learning (RL) has emerged as a potentially transformative technology for the control and optimization of HVAC systems. This project focuses on the use of (distributed) reinforcement learning as a supervisory controller to achieve minimal energy consumption in the cooling system of a data center. The use of distributed reinforcement learning could be highly beneficial, especially considering the large and complex nature of buildings. Separating the building control loops through distributed reinforcement learning offers the potential for optimizing system performance in this complex context. The cooling system is detailed along with its control variables and objectives. An offline model of the real system is available for training and testing the controllers. The training methods, algorithms used, and the various controllers are presented. Multi-Agent Reinforcement Learning (MARL) is compared with singleagent RL to determine the most effective control approach in this context. The different controllers are tested using real load and weather scenarios to evaluate their performance over a year. Significant results are achieved with the single-agent controllers, demonstrating energy savings of up to 16.6% compared to the closed-loop controller for low loads, and up to 3.4% for high loads. However, the multi-agent controllers did not perform well in this particular training setting. Further exploration of alternative training settings or alternative action spaces is required. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9174485
- author
- Donck, Ewoud
- supervisor
- organization
- year
- 2024
- type
- H3 - Professional qualifications (4 Years - )
- subject
- report number
- TFRT-6252
- other publication id
- 0280-5316
- language
- English
- id
- 9174485
- date added to LUP
- 2024-09-16 08:49:22
- date last changed
- 2024-09-16 08:49:22
@misc{9174485,
abstract = {{Buildings worldwide account for roughly 40% of the total energy consumption which boils down to generating 30% of the total carbon dioxide emissions. Heating, Ventilation, and Air Conditioning (HVAC) systems are the largest consumers of energy within these structures. The complexity of building cooling systems poses significant challenges for achieving optimal control. Conventional approaches to improve energy efficiency in buildings rely on high-performance control systems, which demand substantial time and expertise. Reinforcement Learning (RL) has emerged as a potentially transformative technology for the control and optimization of HVAC systems. This project focuses on the use of (distributed) reinforcement learning as a supervisory controller to achieve minimal energy consumption in the cooling system of a data center. The use of distributed reinforcement learning could be highly beneficial, especially considering the large and complex nature of buildings. Separating the building control loops through distributed reinforcement learning offers the potential for optimizing system performance in this complex context. The cooling system is detailed along with its control variables and objectives. An offline model of the real system is available for training and testing the controllers. The training methods, algorithms used, and the various controllers are presented. Multi-Agent Reinforcement Learning (MARL) is compared with singleagent RL to determine the most effective control approach in this context. The different controllers are tested using real load and weather scenarios to evaluate their performance over a year. Significant results are achieved with the single-agent controllers, demonstrating energy savings of up to 16.6% compared to the closed-loop controller for low loads, and up to 3.4% for high loads. However, the multi-agent controllers did not perform well in this particular training setting. Further exploration of alternative training settings or alternative action spaces is required.}},
author = {{Donck, Ewoud}},
language = {{eng}},
note = {{Student Paper}},
title = {{Distributed Reinforcement Learning for Building Energy Optimization}},
year = {{2024}},
}