Performance Analysis with Bayesian Inference
(2023) The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)- Abstract
- Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the... (More)
- Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/a6284fa9-f0d8-42f6-b168-8c26ae9734a3
- author
- Couderc, Noric LU ; Reichenbach, Christoph LU and Söderberg, Emma LU
- organization
- publishing date
- 2023
- type
- Chapter in Book/Report/Conference proceeding
- publication status
- published
- subject
- host publication
- 2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
- publisher
- IEEE - Institute of Electrical and Electronics Engineers Inc.
- conference name
- The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
- conference location
- Melbourne, Australia
- conference dates
- 2023-05-14 - 2023-05-20
- external identifiers
-
- scopus:85172096368
- ISBN
- 979-8-3503-0040-6
- 979-8-3503-0039-0
- DOI
- 10.1109/ICSE-NIER58687.2023.00026
- language
- English
- LU publication?
- yes
- id
- a6284fa9-f0d8-42f6-b168-8c26ae9734a3
- date added to LUP
- 2023-06-27 17:53:17
- date last changed
- 2024-04-19 00:43:12
@inproceedings{a6284fa9-f0d8-42f6-b168-8c26ae9734a3, abstract = {{Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret.}}, author = {{Couderc, Noric and Reichenbach, Christoph and Söderberg, Emma}}, booktitle = {{2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)}}, isbn = {{979-8-3503-0040-6}}, language = {{eng}}, publisher = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}}, title = {{Performance Analysis with Bayesian Inference}}, url = {{http://dx.doi.org/10.1109/ICSE-NIER58687.2023.00026}}, doi = {{10.1109/ICSE-NIER58687.2023.00026}}, year = {{2023}}, }