Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Automatic Differentiation over Fluid Models for Holistic Load Balancing

Heimerson, Albin LU orcid ; Ruuskanen, Johan LU orcid and Eker, Johan LU orcid (2022) p.13-18
Abstract
Microservice applications consist of a set of smaller services interacting in a graph structure to deliver the full application. Jobs will traverse this graph in different paths, both depending on the type of job, but also on the current load of different service replicas. Different paths will incur different scenario-specific costs, dependent on, e.g., deployment and the underlying cloud system. In this paper, we demonstrate how automatic differentiation over data-driven fluid models can be used to optimize a running microservice application, by designing a load balancer that minimizes some holistic cost function under response time percentile constraints. The cost function is based on performance metrics from a fluid model retrieved... (More)
Microservice applications consist of a set of smaller services interacting in a graph structure to deliver the full application. Jobs will traverse this graph in different paths, both depending on the type of job, but also on the current load of different service replicas. Different paths will incur different scenario-specific costs, dependent on, e.g., deployment and the underlying cloud system. In this paper, we demonstrate how automatic differentiation over data-driven fluid models can be used to optimize a running microservice application, by designing a load balancer that minimizes some holistic cost function under response time percentile constraints. The cost function is based on performance metrics from a fluid model retrieved through logs from the application. The gradient of this cost, with respect to the load balancing parameters, is calculated via automatic differentiation. This enables parameter updates, using e.g. gradient descent, that steers the application towards a setting of less cost. In an experimental evaluation on a small microservice application running on Ericsson Research Datacenter, it is shown that the method can quickly step towards optimal values while supporting complicated cost functions such as solutions to a system of ordinary differential equations. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)
pages
13 - 18
external identifiers
  • scopus:85143079123
ISBN
978-1-6654-5142-0
DOI
10.1109/ACSOSC56246.2022.00020
project
Event-Based Information Fusion for the Self-Adaptive Cloud
Event-Based Control of Stochastic Systems with Application to Server Systems
language
English
LU publication?
yes
id
ccdaf1d2-1d2d-45b9-b1c1-4c86fe8ca45f
date added to LUP
2022-11-21 11:05:28
date last changed
2023-11-21 13:00:16
@inproceedings{ccdaf1d2-1d2d-45b9-b1c1-4c86fe8ca45f,
  abstract     = {{Microservice applications consist of a set of smaller services interacting in a graph structure to deliver the full application. Jobs will traverse this graph in different paths, both depending on the type of job, but also on the current load of different service replicas. Different paths will incur different scenario-specific costs, dependent on, e.g., deployment and the underlying cloud system. In this paper, we demonstrate how automatic differentiation over data-driven fluid models can be used to optimize a running microservice application, by designing a load balancer that minimizes some holistic cost function under response time percentile constraints. The cost function is based on performance metrics from a fluid model retrieved through logs from the application. The gradient of this cost, with respect to the load balancing parameters, is calculated via automatic differentiation. This enables parameter updates, using e.g. gradient descent, that steers the application towards a setting of less cost. In an experimental evaluation on a small microservice application running on Ericsson Research Datacenter, it is shown that the method can quickly step towards optimal values while supporting complicated cost functions such as solutions to a system of ordinary differential equations.}},
  author       = {{Heimerson, Albin and Ruuskanen, Johan and Eker, Johan}},
  booktitle    = {{2022 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)}},
  isbn         = {{978-1-6654-5142-0}},
  language     = {{eng}},
  month        = {{11}},
  pages        = {{13--18}},
  title        = {{Automatic Differentiation over Fluid Models for Holistic Load Balancing}},
  url          = {{https://lup.lub.lu.se/search/files/141037075/ACSOS_2022_Dynorc_No_template_.pdf}},
  doi          = {{10.1109/ACSOSC56246.2022.00020}},
  year         = {{2022}},
}