Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

Exploring the performance of ML model size for classification in relation to energy consumption

Bexell, Andreas LU ; Gullstrand Heander, Lo LU orcid ; Söderberg, Emma LU orcid ; Eldh, Sigrid and Runeson, Per LU orcid (2025) In Lecture Notes in Computer Science 16361. p.525-532
Abstract
The use of large language models (LLMs) is being explored for a multitude of tasks in software engineering (SE), ranging from code generation to bug report assignment. Although LLMs provide impressive results, they require more time and energy than some other machine learning models. For some tasks, simpler models may be more sustainable than LLMs.

In this paper, we construct natural language classifiers of different complexity for a use case in the SE domain: commit message classification. We compare the performance of each model with the state-of-the-art with regard to energy consumption for training and inference. We find that simpler models based on Naïve Bayes and LSTM
perform similarly to LLMs, while using a fraction... (More)
The use of large language models (LLMs) is being explored for a multitude of tasks in software engineering (SE), ranging from code generation to bug report assignment. Although LLMs provide impressive results, they require more time and energy than some other machine learning models. For some tasks, simpler models may be more sustainable than LLMs.

In this paper, we construct natural language classifiers of different complexity for a use case in the SE domain: commit message classification. We compare the performance of each model with the state-of-the-art with regard to energy consumption for training and inference. We find that simpler models based on Naïve Bayes and LSTM
perform similarly to LLMs, while using a fraction of the energy, suggesting that choosing a small model can lead to significant reduction in power usage without compromising performance.

Replication package: https://doi.org/10.5281/zenodo.15641782 (Less)
Please use this url to cite or link to this publication:
author
; ; ; and
organization
publishing date
type
Chapter in Book/Report/Conference proceeding
publication status
published
subject
host publication
Product-focused software process improvement : 26th International Conference, PROFES 2025, Salerno, Italy, December 1–3, 2025 - 26th International Conference, PROFES 2025, Salerno, Italy, December 1–3, 2025
series title
Lecture Notes in Computer Science
editor
Scanniello, Giuseppe ; Lenarduzzi, Valentina ; Romano, Simone ; Vegas, Sira and Francese, Rita
volume
16361
pages
525 - 532
publisher
Springer
external identifiers
  • scopus:105023309214
ISSN
1611-3349
ISBN
978-3-032-12089-2
978-3-032-12088-5
DOI
10.1007/978-3-032-12089-2_38
language
English
LU publication?
yes
id
8596bf17-0db4-4694-aafd-3ad33c1e49ed
date added to LUP
2025-11-25 10:44:29
date last changed
2025-12-05 04:00:09
@inproceedings{8596bf17-0db4-4694-aafd-3ad33c1e49ed,
  abstract     = {{The use of large language models (LLMs) is being explored for a multitude of tasks in software engineering (SE), ranging from code generation to bug report assignment. Although LLMs provide impressive results, they require more time and energy than some other machine learning models. For some tasks, simpler models may be more sustainable than LLMs. <br/><br/>In this paper, we construct natural language classifiers of different complexity for a use case in the SE domain: commit message classification. We compare the performance of each model with the state-of-the-art with regard to energy consumption for training and inference. We find that simpler models based on Naïve Bayes and LSTM <br/>perform similarly to LLMs, while using a fraction of the energy, suggesting that choosing a small model can lead to significant reduction in power usage without compromising performance.<br/><br/>Replication package: https://doi.org/10.5281/zenodo.15641782}},
  author       = {{Bexell, Andreas and Gullstrand Heander, Lo and Söderberg, Emma and Eldh, Sigrid and Runeson, Per}},
  booktitle    = {{Product-focused software process improvement : 26th International Conference, PROFES 2025, Salerno, Italy, December 1–3, 2025}},
  editor       = {{Scanniello, Giuseppe and Lenarduzzi, Valentina and Romano, Simone and Vegas, Sira and Francese, Rita}},
  isbn         = {{978-3-032-12089-2}},
  issn         = {{1611-3349}},
  language     = {{eng}},
  month        = {{11}},
  pages        = {{525--532}},
  publisher    = {{Springer}},
  series       = {{Lecture Notes in Computer Science}},
  title        = {{Exploring the performance of ML model size for classification in relation to energy consumption}},
  url          = {{http://dx.doi.org/10.1007/978-3-032-12089-2_38}},
  doi          = {{10.1007/978-3-032-12089-2_38}},
  volume       = {{16361}},
  year         = {{2025}},
}