Conform with the Wind : Processing short-term ensemble forecasts with conformal based methods for probabilistic wind-speed forecasting
(2023) In Master's Theses in Mathematical Sciences FMSM01 20231Mathematical Statistics
- Abstract
- Forecasting wind has always been an interesting subject, and as large parts of the world are relying more on wind for power production it is becoming even more important to have reliable forecasts. Probabilistic forecasts, where distributions are predicted in contrast to deterministic forecasts, are impor- tant for informed decision making. We apply two methods based on conformal prediction for processing ensemble forecasts to well calibrated probability dis- tributions. Conformal prediction is a relatively modern method for quantified uncertainty analysis within machine learning. These methods are compared to the quantile regression forest algorithm, which has been well tested in literature for probabilistic ensemble post processing.... (More)
- Forecasting wind has always been an interesting subject, and as large parts of the world are relying more on wind for power production it is becoming even more important to have reliable forecasts. Probabilistic forecasts, where distributions are predicted in contrast to deterministic forecasts, are impor- tant for informed decision making. We apply two methods based on conformal prediction for processing ensemble forecasts to well calibrated probability dis- tributions. Conformal prediction is a relatively modern method for quantified uncertainty analysis within machine learning. These methods are compared to the quantile regression forest algorithm, which has been well tested in literature for probabilistic ensemble post processing. Ensemble forecasting is a method based on running several numerical weather prediction models simultaneously, creating an array of forecasts. The conformal methods rely on an additional point forecast, supplied by another model, for producing the distributions while the quantile regression forest works directly on the ensemble. The methods were tested using a teaching schedule which determines the best configuration of parameters, from a predefined set, based on the continuous ranked proba- bility score metric before making each prediction. This is a way of simulating how the methods perform over time. For the conformal methods we employ a normalized version of conformal predictive distribution systems and a non- exchangeable conformal prediction method. For the non-exchangeable case we suggest a method of stacking confidence intervals to produce distributions. We also suggest a normalized version of this algorithm. Both methods show promis- ing results, both able to produce significantly better distributions than the raw ensemble and as good or better calibrated distributions compared to the quantile regression forest. Though the conformal methods are supplied external forecasts and the quantile regression forest is not using an optimal configuration. The conformal methods also produce well calibrated predictions consistently over different setups of the algorithms. (Less)
- Popular Abstract
- Creating better and more applicable wind forecasts
From propelling sailing vessels for centuries to driving wind turbines in modern times, wind is continuously very important to our society. The importance of weather to us humans is exemplified by the fact that we have been trying to predict it for thousands of years. In recent times the fields of statistics and machine learning have developed methods that allow for much better forecasts. One such method, Conformal Prediction, shows a lot of promise for producing reliable probabilistic forecasts for wind. These forecasts take the form of probability distributions, which gives a wider picture of the potential weather than traditional forecasts.
From the year 2000 to 2020 the amount... (More) - Creating better and more applicable wind forecasts
From propelling sailing vessels for centuries to driving wind turbines in modern times, wind is continuously very important to our society. The importance of weather to us humans is exemplified by the fact that we have been trying to predict it for thousands of years. In recent times the fields of statistics and machine learning have developed methods that allow for much better forecasts. One such method, Conformal Prediction, shows a lot of promise for producing reliable probabilistic forecasts for wind. These forecasts take the form of probability distributions, which gives a wider picture of the potential weather than traditional forecasts.
From the year 2000 to 2020 the amount of power generated from wind in Sweden increased with a factor more than 50. As a society we rely more and more on wind for our energy needs which also increases our reliance on forecasts. Wind is not constant so to ensure that we have stable power grids we must plan production of electricity from other sources. Moving from traditional forecasts, which gives a single value for the wind, to probabilistic forecasts gives more flexibility for decision making. There are several methods for making probabilistic forecasts from the fields of Statistics and Machine Learning. The methods tested in this study are versions of Conformal Prediction which is relatively new and unexplored within meteorology.
The methods show promising results in their ability to create good probability distributions. The distributions are good in the sense that they seem to match the real distribution the wind might have. Compared to a more established method, the Quantile Regression Forest, the Conformal methods perform well and requires far less computing power to produce good results. The new methods are yet to be tested against an optimized version of the Quantile Regression Forest. However, the Conformal Methods also have a lot of potential for improvement.
The effects of these results are many. One possibility is to supplement local forecasts with probability distributions through these methods. This could be very useful for not only power production but also planning of aviation, seafaring and similar activities.
Conformal prediction is a method that has been developed since the late 90s but has risen in popularity the last couple of years. It is still relatively unknown compared to many similar methods. The unique part of Conformal prediction is that it gives theoretical guarantees that the predictions are correct. It is also a very straight forward method that still allows a lot of flexibility. Meaning it can be modified to fit most applications.
Conformal prediction has a clear parallel to a process we might do in our daily lives to form a picture of how sure we are of a prediction. Suppose you want to predict the temperature outside by looking out of your windows. It might be sunny, and you would then naturally predict that it is warm. Suppose you go outside and this is the case, you would then take a mental note that the weather behaved as you expected. That it conforms to what you have experienced previously. Suppose the opposite happens, that it is cold, and you take a mental note that it in this case it behaved strangely. With enough of these notes of how often and how much each situation conforms to your experience, you would gain a better understanding what the possible ranges of temperature are in each case.
The methods in this study were tested on wind measurements and forecasts over a year on the west coast of Sweden. They predicted the wind speed 24 hours before noon each day. Before making a prediction, each method was modified to perform as well as possible by choosing from a small set of configurations. There is a lot of research still to go before any definitive claims about the usefulness of the Conformal methods can be claimed. More test-data in more locations and comparing against more optimized methods are the first steps. However, these first results are very promising. (Less)
Please use this url to cite or link to this publication:
http://lup.lub.lu.se/student-papers/record/9130213
- author
- Althoff, Simon LU
- supervisor
- organization
- alternative title
- Foga sig med vinden: efterbehandling av närtida ensembleprognoser genom conformal-baserade metoder för probabilistiska vindprognoser
- course
- FMSM01 20231
- year
- 2023
- type
- H2 - Master's Degree (Two Years)
- subject
- keywords
- wind-speed, probabilistic, forecasts, forecasting, ensemble, post-processing, conformal prediction, conformal predictive systems, quantile regression forest, non-exchangeable
- publication/series
- Master's Theses in Mathematical Sciences
- report number
- LUTFMS-3488-2023
- ISSN
- 1404-6342
- other publication id
- 2023:E61
- language
- English
- id
- 9130213
- date added to LUP
- 2023-06-26 14:11:52
- date last changed
- 2023-07-03 14:35:39
@misc{9130213,
abstract = {{Forecasting wind has always been an interesting subject, and as large parts of the world are relying more on wind for power production it is becoming even more important to have reliable forecasts. Probabilistic forecasts, where distributions are predicted in contrast to deterministic forecasts, are impor- tant for informed decision making. We apply two methods based on conformal prediction for processing ensemble forecasts to well calibrated probability dis- tributions. Conformal prediction is a relatively modern method for quantified uncertainty analysis within machine learning. These methods are compared to the quantile regression forest algorithm, which has been well tested in literature for probabilistic ensemble post processing. Ensemble forecasting is a method based on running several numerical weather prediction models simultaneously, creating an array of forecasts. The conformal methods rely on an additional point forecast, supplied by another model, for producing the distributions while the quantile regression forest works directly on the ensemble. The methods were tested using a teaching schedule which determines the best configuration of parameters, from a predefined set, based on the continuous ranked proba- bility score metric before making each prediction. This is a way of simulating how the methods perform over time. For the conformal methods we employ a normalized version of conformal predictive distribution systems and a non- exchangeable conformal prediction method. For the non-exchangeable case we suggest a method of stacking confidence intervals to produce distributions. We also suggest a normalized version of this algorithm. Both methods show promis- ing results, both able to produce significantly better distributions than the raw ensemble and as good or better calibrated distributions compared to the quantile regression forest. Though the conformal methods are supplied external forecasts and the quantile regression forest is not using an optimal configuration. The conformal methods also produce well calibrated predictions consistently over different setups of the algorithms.}},
author = {{Althoff, Simon}},
issn = {{1404-6342}},
language = {{eng}},
note = {{Student Paper}},
series = {{Master's Theses in Mathematical Sciences}},
title = {{Conform with the Wind : Processing short-term ensemble forecasts with conformal based methods for probabilistic wind-speed forecasting}},
year = {{2023}},
}