Advanced

High-risk Consumer Credit Scoring using Machine Learning Classification

Mjörnell, Max and Levay, Ludvig (2019) In LUTFMS-3389-2020 FMSM01 20191
Mathematical Statistics
Abstract
The use of statistical models in credit rating and application scorecard modelling is a thoroughly explored field within the financial sector and a central component in a credit institution’s underlying business model. The aim of this report was to apply and compare six different machine learning models in predicting credit defaults for high-risk consumer credits, using a data set provided by a
Swedish consumer credit institute. The selected models include the ones most frequently used for scorecard modelling across the banking industry as well as some more rarely used that could potentially add valuable insights. The models are briefly introduced and the most important concepts for each model are explained, as well as how to deal with... (More)
The use of statistical models in credit rating and application scorecard modelling is a thoroughly explored field within the financial sector and a central component in a credit institution’s underlying business model. The aim of this report was to apply and compare six different machine learning models in predicting credit defaults for high-risk consumer credits, using a data set provided by a
Swedish consumer credit institute. The selected models include the ones most frequently used for scorecard modelling across the banking industry as well as some more rarely used that could potentially add valuable insights. The models are briefly introduced and the most important concepts for each model are explained, as well as how to deal with the lack of transparency in complex models by the use of white-boxing methods. Appropriate metrics for evaluating prediction performance on imbalanced and insufficient data sets are discussed, as well as how to increase model performance by using different oversampling techniques. All available information about the loan applicants was then exhaustively examined, and a carefully refined set of input features was constructed to ensure optimal predictive power and generalizability. After tuning and testing the models, the results showed that logistic regression, support vector machine, neural network and a soft voting ensemble showed similar performance results using the same input feature configurations. Attempts to create synthesized samples to handle the imbalance problem showed no effect and was therefore not used. The white-boxing model SHAP showed a promising ability to instructively explain the underlying decision basis for complex models such as neural networks. However, considering the limited data set at hand, the recommended model to use is the logistic regression model given its simplicity and on a par performance with the other models. Having larger amounts of data available on the other hand, the more complex models such as neural networks and support vector machines could have a potential advantage. (Less)
Popular Abstract
Machine learning has recently become the word on everyone’s lips, and its application areas are currently booming. Financial institutions are frequently using new discoveries to automate and optimize their businesses, including the issue of rating loan applicants’ creditworthiness. The most widely used approach today is either using a simple statistical model or judgmental decisionmaking, where credit analysts use prior knowledge to predict whether a loan will default or not.
In this report, some of the most popular machine learning algorithms are successfully applied on the area of application scorecard modelling. The aim is to investigate if cutting-edge models could outperform more outdated approaches such as logistic regression on a... (More)
Machine learning has recently become the word on everyone’s lips, and its application areas are currently booming. Financial institutions are frequently using new discoveries to automate and optimize their businesses, including the issue of rating loan applicants’ creditworthiness. The most widely used approach today is either using a simple statistical model or judgmental decisionmaking, where credit analysts use prior knowledge to predict whether a loan will default or not.
In this report, some of the most popular machine learning algorithms are successfully applied on the area of application scorecard modelling. The aim is to investigate if cutting-edge models could outperform more outdated approaches such as logistic regression on a relatively small data set to gain knowledge from. In addition, techniques to increase human interaction was implemented in
order for a human to understand complex machine learning classifications. The results showed that complex models such as an artificial neural network did not top the simpler models, and the reason is believed to be the limited amount of available data. Given more data on the other hand, the more sophisticated machine learning classifiers are believed to have a prosperous future in credit
scoring. They were also successfully deciphered, so a human easily could interpret even the most complicated operation. The resulting models have the potential to significantly decrease default rates if implemented, and in the long run lead to increased profitability as well as fewer people ending up in mountains of debt they can’t repay. (Less)
Please use this url to cite or link to this publication:
author
Mjörnell, Max and Levay, Ludvig
supervisor
organization
course
FMSM01 20191
year
type
H2 - Master's Degree (Two Years)
subject
keywords
Machine learning, Scorecard modelling, Logistic regression, Support Vector Machine, Decision Tree, Random Forest, k-Nearest Neighbors, Artificial Neural Network, Voting ensemble, SHAP, LIME, Average Precision score, Feature engineering
publication/series
LUTFMS-3389-2020
report number
2020:E14
ISSN
1404-6342
language
English
additional info
Uppladdad i efterhand mars 2020. Arbetet avslutades i juni 2019.
id
9006336
date added to LUP
2020-03-10 16:01:26
date last changed
2020-03-10 16:01:26
@misc{9006336,
  abstract     = {The use of statistical models in credit rating and application scorecard modelling is a thoroughly explored field within the financial sector and a central component in a credit institution’s underlying business model. The aim of this report was to apply and compare six different machine learning models in predicting credit defaults for high-risk consumer credits, using a data set provided by a
Swedish consumer credit institute. The selected models include the ones most frequently used for scorecard modelling across the banking industry as well as some more rarely used that could potentially add valuable insights. The models are briefly introduced and the most important concepts for each model are explained, as well as how to deal with the lack of transparency in complex models by the use of white-boxing methods. Appropriate metrics for evaluating prediction performance on imbalanced and insufficient data sets are discussed, as well as how to increase model performance by using different oversampling techniques. All available information about the loan applicants was then exhaustively examined, and a carefully refined set of input features was constructed to ensure optimal predictive power and generalizability. After tuning and testing the models, the results showed that logistic regression, support vector machine, neural network and a soft voting ensemble showed similar performance results using the same input feature configurations. Attempts to create synthesized samples to handle the imbalance problem showed no effect and was therefore not used. The white-boxing model SHAP showed a promising ability to instructively explain the underlying decision basis for complex models such as neural networks. However, considering the limited data set at hand, the recommended model to use is the logistic regression model given its simplicity and on a par performance with the other models. Having larger amounts of data available on the other hand, the more complex models such as neural networks and support vector machines could have a potential advantage.},
  author       = {Mjörnell, Max and Levay, Ludvig},
  issn         = {1404-6342},
  keyword      = {Machine learning,Scorecard modelling,Logistic regression,Support Vector Machine,Decision Tree,Random Forest,k-Nearest Neighbors,Artificial Neural Network,Voting ensemble,SHAP,LIME,Average Precision score,Feature engineering},
  language     = {eng},
  note         = {Student Paper},
  series       = {LUTFMS-3389-2020},
  title        = {High-risk Consumer Credit Scoring using Machine Learning Classification},
  year         = {2019},
}