Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Performance Analysis with Bayesian Inference

Couderc, Noric LU orcid ; Reichenbach, Christoph LU orcid and Söderberg, Emma LU orcid (2023) The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
Abstract
Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the... (More)
Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret. (Less)
Please use this url to cite or link to this publication:
author
; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
publisher
IEEE - Institute of Electrical and Electronics Engineers Inc.
conference name
The 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
conference location
Melbourne, Australia
conference dates
2023-05-14 - 2023-05-20
external identifiers
  • scopus:85172096368
ISBN
979-8-3503-0040-6
979-8-3503-0039-0
DOI
10.1109/ICSE-NIER58687.2023.00026
language
English
LU publication?
yes
id
a6284fa9-f0d8-42f6-b168-8c26ae9734a3
date added to LUP
2023-06-27 17:53:17
date last changed
2024-04-19 00:43:12
@inproceedings{a6284fa9-f0d8-42f6-b168-8c26ae9734a3,
  abstract     = {{Statistics are part of any empirical science, and performance analysis is no exception. However, for non-statisticians, picking the right statistical tool to answer a research question can be challenging; each statistical tool comes with a set of assumptions, and it is not clear to researchers what happens when those assumptions are violated. Bayesian statistics offers a framework with more flexibility and with explicit assumptions. In this paper, we present a method to analyse benchmark results using Bayesian inference. We demonstrate how to perform a Bayesian analysis of variance (ANOVA) to estimate what factors matter most for performance, and describe how to investigate what factors affect the impact of optimizations. We find the Bayesian model more flexible, and the Bayesian ANOVA’s output easier to interpret.}},
  author       = {{Couderc, Noric and Reichenbach, Christoph and Söderberg, Emma}},
  booktitle    = {{2023 IEEE/ACM 45th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)}},
  isbn         = {{979-8-3503-0040-6}},
  language     = {{eng}},
  publisher    = {{IEEE - Institute of Electrical and Electronics Engineers Inc.}},
  title        = {{Performance Analysis with Bayesian Inference}},
  url          = {{http://dx.doi.org/10.1109/ICSE-NIER58687.2023.00026}},
  doi          = {{10.1109/ICSE-NIER58687.2023.00026}},
  year         = {{2023}},
}