Impact of etcd deployment on Kubernetes, Istio, and application performance
(2020) In Software - Practice and Experience- Abstract
- This experience article describes lessons learned as we conducted experiments in a Kubernetes‐based environment, the most notable of which was that the performance of both the Kubernetes control plane and the deployed application depends strongly and in unexpected ways on the performance of the etcd database. The article contains (a) detailed descriptions of how networking with and without Istio works in Kubernetes, based on the Flannel Container Networking Interface (CNI) provider in VXLAN mode with IP Virtual Server (IPVS)‐backed Kubernetes Services, (b) a comprehensive discussion about how to conduct load and performance testing using a closed‐loop workload generator, and (c) an open source experiment framework useful for executing... (More)
- This experience article describes lessons learned as we conducted experiments in a Kubernetes‐based environment, the most notable of which was that the performance of both the Kubernetes control plane and the deployed application depends strongly and in unexpected ways on the performance of the etcd database. The article contains (a) detailed descriptions of how networking with and without Istio works in Kubernetes, based on the Flannel Container Networking Interface (CNI) provider in VXLAN mode with IP Virtual Server (IPVS)‐backed Kubernetes Services, (b) a comprehensive discussion about how to conduct load and performance testing using a closed‐loop workload generator, and (c) an open source experiment framework useful for executing experiments in a shared cloud environment and exploring the resulting data. It also shows that statistical analysis may reveal the data resulting from such experiments to be misleading even when careful preparations are made, and that nondeterministic behavior stemming from etcd can affect both the platform as a whole and the deployed application. Finally, it is demonstrated that using high‐performance backing storage for etcd can reduce the occurrence of such nondeterministic behaviors by a statistically significant (P < .05) margin. The implication of this experience article is that systems researchers studying the performance of applications deployed on Kubernetes cannot simply consider their specific application to be under test. Instead, the particularities of the underlying Kubernetes and cloud platform must be taken into account, in particular because their performance can impact that of etcd. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/371464ed-516d-4d89-9eec-4f4efc352a78
- author
- Larsson, Lars ; Tärneberg, William LU ; Klein, Cristian ; Elmroth, Erik and Kihl, Maria LU
- organization
- publishing date
- 2020-08-07
- type
- Contribution to journal
- publication status
- published
- subject
- in
- Software - Practice and Experience
- publisher
- John Wiley & Sons Inc.
- external identifiers
-
- scopus:85089109897
- ISSN
- 0038-0644
- DOI
- 10.1002/spe.2885
- project
- Cyber Security for Next Generation Factory (SEC4FACTORY)
- language
- English
- LU publication?
- yes
- id
- 371464ed-516d-4d89-9eec-4f4efc352a78
- date added to LUP
- 2020-06-11 13:08:26
- date last changed
- 2023-04-10 14:45:49
@article{371464ed-516d-4d89-9eec-4f4efc352a78,
abstract = {{This experience article describes lessons learned as we conducted experiments in a Kubernetes‐based environment, the most notable of which was that the performance of both the Kubernetes control plane and the deployed application depends strongly and in unexpected ways on the performance of the etcd database. The article contains (a) detailed descriptions of how networking with and without Istio works in Kubernetes, based on the Flannel Container Networking Interface (CNI) provider in VXLAN mode with IP Virtual Server (IPVS)‐backed Kubernetes Services, (b) a comprehensive discussion about how to conduct load and performance testing using a closed‐loop workload generator, and (c) an open source experiment framework useful for executing experiments in a shared cloud environment and exploring the resulting data. It also shows that statistical analysis may reveal the data resulting from such experiments to be misleading even when careful preparations are made, and that nondeterministic behavior stemming from etcd can affect both the platform as a whole and the deployed application. Finally, it is demonstrated that using high‐performance backing storage for etcd can reduce the occurrence of such nondeterministic behaviors by a statistically significant (P < .05) margin. The implication of this experience article is that systems researchers studying the performance of applications deployed on Kubernetes cannot simply consider their specific application to be under test. Instead, the particularities of the underlying Kubernetes and cloud platform must be taken into account, in particular because their performance can impact that of etcd.}},
author = {{Larsson, Lars and Tärneberg, William and Klein, Cristian and Elmroth, Erik and Kihl, Maria}},
issn = {{0038-0644}},
language = {{eng}},
month = {{08}},
publisher = {{John Wiley & Sons Inc.}},
series = {{Software - Practice and Experience}},
title = {{Impact of etcd deployment on Kubernetes, Istio, and application performance}},
url = {{http://dx.doi.org/10.1002/spe.2885}},
doi = {{10.1002/spe.2885}},
year = {{2020}},
}