Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Adaptive Control of Data Center Cooling using Deep Reinforcement Learning

Heimerson, Albin LU orcid ; Sjölund, Johannes ; Brännvall, Rickard ; Gustafsson, Jonas and Eker, Johan LU orcid (2022) p.1-6
Abstract
In this paper, we explore the use of Reinforcement Learning (RL) to improve the control of cooling equipment in Data Centers (DCs). DCs are inherently complex systems, and thus challenging to model from first principles. Machine learning offers a way to address this by instead training a model to capture the thermal dynamics of a DC. In RL, an agent learns to control a system through trial-and-error. However, for systems such as DCs, an interactive trial-and-error approach is not possible, and instead, a high-fidelity model is needed. In this paper, we develop a DC model using Computational Fluid Dynamics (CFD) based on the Lattice Boltzmann Method (LBM) Bhatnagar-Gross-Krook (BGK) algorithm. The model features transient boundary... (More)
In this paper, we explore the use of Reinforcement Learning (RL) to improve the control of cooling equipment in Data Centers (DCs). DCs are inherently complex systems, and thus challenging to model from first principles. Machine learning offers a way to address this by instead training a model to capture the thermal dynamics of a DC. In RL, an agent learns to control a system through trial-and-error. However, for systems such as DCs, an interactive trial-and-error approach is not possible, and instead, a high-fidelity model is needed. In this paper, we develop a DC model using Computational Fluid Dynamics (CFD) based on the Lattice Boltzmann Method (LBM) Bhatnagar-Gross-Krook (BGK) algorithm. The model features transient boundary conditions for simulating the DC room, heat-generating servers, and Computer Room Air Handlers (CRAHs) as well as rejection components outside the server room such as heat exchangers, compressors, and dry coolers. This model is used to train an RL agent to control the cooling equipment. Evaluations show that the RL agent can outperform traditional controllers and also can adapt to changes in the environment, such as equipment breaking down. (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)
pages
1 - 6
external identifiers
  • scopus:85143075382
DOI
10.1109/ACSOSC56246.2022.00018
project
Event-Based Control of Stochastic Systems with Application to Server Systems
AutoDC
language
English
LU publication?
yes
id
aa5642ed-9798-4451-bed3-465e0bea7edc
date added to LUP
2022-11-21 11:09:32
date last changed
2023-11-21 13:00:16
@inproceedings{aa5642ed-9798-4451-bed3-465e0bea7edc,
  abstract     = {{In this paper, we explore the use of Reinforcement Learning (RL) to improve the control of cooling equipment in Data Centers (DCs). DCs are inherently complex systems, and thus challenging to model from first principles. Machine learning offers a way to address this by instead training a model to capture the thermal dynamics of a DC. In RL, an agent learns to control a system through trial-and-error. However, for systems such as DCs, an interactive trial-and-error approach is not possible, and instead, a high-fidelity model is needed. In this paper, we develop a DC model using Computational Fluid Dynamics (CFD) based on the Lattice Boltzmann Method (LBM) Bhatnagar-Gross-Krook (BGK) algorithm. The model features transient boundary conditions for simulating the DC room, heat-generating servers, and Computer Room Air Handlers (CRAHs) as well as rejection components outside the server room such as heat exchangers, compressors, and dry coolers. This model is used to train an RL agent to control the cooling equipment. Evaluations show that the RL agent can outperform traditional controllers and also can adapt to changes in the environment, such as equipment breaking down.}},
  author       = {{Heimerson, Albin and Sjölund, Johannes and Brännvall, Rickard and Gustafsson, Jonas and Eker, Johan}},
  booktitle    = {{2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)}},
  language     = {{eng}},
  month        = {{11}},
  pages        = {{1--6}},
  title        = {{Adaptive Control of Data Center Cooling using Deep Reinforcement Learning}},
  url          = {{https://lup.lub.lu.se/search/files/141036901/ACSOS_2022_RafsineRL_No_template_.pdf}},
  doi          = {{10.1109/ACSOSC56246.2022.00018}},
  year         = {{2022}},
}