Functional ANOVA for Inherently Interpretable Gradient Boosting in PD Modelling

Abstract

For financial institutions, estimating the probability of default is central to credit-risk modelling. This is traditionally done using logistic regression, but machine learning methods have shown promising results because of their ability to capture more flexible relationships. However, the complexity of such models makes them difficult to interpret and consequently difficult to justify to regulators.

This thesis explores the use of functional ANOVA decomposition to construct an interpretable representation of depth-limited gradient-boosted decision trees for credit-risk modelling. The aim is to evaluate whether the resulting decomposition can be regarded as an inherently interpretable model that aligns with regulatory expectations.

The results show that the approach successfully reconstructs the fitted model as a sum of main effects and pairwise interaction effects. Simulations showed strong recovery for both main and interaction effects when the target function is observed directly. Recovery is weaker when the target is a latent score function, but the method still approximately recovers the main effects.

The method is also applied to Polish bankruptcy data, where it provides an interpretable representation of a fitted credit-risk model. The decomposition makes it possible to inspect the effects of individual financial ratios and selected pairwise interactions, and to assess whether these effects are reasonable from an economic perspective. The results indicate that the main effects are more stable and easier to interpret than the interaction effects. Overall, the findings suggest that the proposed decomposition can provide a transparent way to use flexible machine learning models in credit-risk modelling by providing a representation that can be directly inspected and interpreted.

Popular Abstract

Can we create a statistical model that is as flexible as possible but also easy to interpret?

That is a question of central importance within predictive modelling, particularly in high-stakes contexts. For financial institutions, for example, estimating the probability of default is of high importance to their lending activities. This is typically accomplished using logistic regression, which is a statistical modelling technique with a long history. Logistic regression is used for a reason. It can achieve good results and the model is easy to use and understand. However, one drawback is that it requires assumptions to be made regarding the structure of the relationships between the variables and the risk before the model is built.

In order to be able to capture any type of structure, we would prefer to make as few assumptions as possible. This is one reason to use more recent modelling techniques, known as machine learning. Machine learning models are characterized by being highly flexible and requiring fewer assumptions compared to traditional methods. However, one drawback is that they are highly complex and difficult to understand, to the point that they are sometimes referred to as "black boxes". Credit-risk modelling is highly regulated, with strict requirements regarding predictive models. The lack of interpretability of machine learning models has therefore led to limited uptake within the industry.

However, there are certain ways to make a machine learning model easier to understand. Often this is done using what is referred to as post-hoc methods, but such methods have the downside of only summarizing what the model does, without allowing us to truly inspect it. We would therefore prefer to have what is known as an inherently interpretable model, which can be understood directly. One such way is through functional decomposition, where we attempt to re-express our model as a sum of functions representing main and interaction effects. However, in order for us to use this sum to interpret the model, we need the decomposition to be unique. One way to accomplish this is to use functional ANOVA, where we choose the functions so that as much as possible is explained by the main effects.

The functional ANOVA decomposition is generally difficult to compute, but if we combine it with a certain category of machine learning models, known as tree ensembles, this can be performed in a relatively fast and efficient way. By doing this, we can construct a flexible machine learning model that is at the same time easier to understand. For credit-risk modelling, this means that we can examine which risk drivers are most important, how they affect the predicted risk, and whether these effects are reasonable from an economic perspective. In this thesis, the method is studied using both simulated data and data on bankruptcies among Polish companies.

The results show that machine learning models do not have to be treated as black boxes. By combining tree-based machine learning with functional ANOVA, it is possible to construct a flexible model that is also interpretable. This suggests that the method has the potential to allow for a more flexible modelling approach for credit-risk modelling that aligns with legal requirements.

• author: Kopelman, Sara
• supervisor: Magnus Wiktorsson
• organization: Mathematical Statistics
• course: MASM02 20261
• year: 2026
• type: H2 - Master's Degree (Two Years)
• subject: Mathematics and Statistics
• keywords: probability of default, credit-risk, gradient-boosting, decision trees, functional ANOVA, interpretability, machine learning
• publication/series: Master's Theses in Mathematical Sciences
• report number: LUNFMS-3143-2026
• ISSN: 1404-6342
• other publication id: 2026:E44
• language: English
• id: 9242278
• date added to LUP: 2026-06-22 16:41:11
• date last changed: 2026-06-22 16:41:11