Species distributions models may predict accurately future distributions but poorly how distributions change : A critical perspective on model validation
Piirainen, Sirke ; Lehikoinen, Aleksi LU ; Husby, Magne ; Kålås, John Atle ; Lindström, Åke LU
and Ovaskainen, Otso
(2023)
In Diversity and Distributions
29(5).
p.654-665
- Abstract
Aim: Species distribution models (SDMs) are widely used to make predictions on how species distributions may change as a response to climatic change. To assess the reliability of those predictions, they need to be critically validated with respect to what they are used for. While ecologists are typically interested in how and where distributions will change, we argue that SDMs have seldom been evaluated in terms of their capacity to predict such change. Instead, typical retrospective validation methods estimate model's ability to predict to only one static time in future. Here, we apply two validation methods, one that predicts and evaluates a static pattern, while the other measures change and compare their estimates of predictive... (More)
Aim: Species distribution models (SDMs) are widely used to make predictions on how species distributions may change as a response to climatic change. To assess the reliability of those predictions, they need to be critically validated with respect to what they are used for. While ecologists are typically interested in how and where distributions will change, we argue that SDMs have seldom been evaluated in terms of their capacity to predict such change. Instead, typical retrospective validation methods estimate model's ability to predict to only one static time in future. Here, we apply two validation methods, one that predicts and evaluates a static pattern, while the other measures change and compare their estimates of predictive performance. Location: Fennoscandia. Methods: We applied a joint SDM to model the distributions of 120 bird species in four model validation settings. We trained models with a dataset from 1975 to 1999 and predicted species' future occurrence and abundance in two ways: for one static time period (2013–2016, ‘static validation’) and for a change between two time periods (difference between 1996–1999 and 2013–2016, ‘change validation’). We then measured predictive performance using correlation between predicted and observed values. We also related predictive performance to species traits. Results: Even though static validation method evaluated predictive performance as good, change method indicated very poor performance. Predictive performance was not strongly related to any trait. Main Conclusions: Static validation method might overestimate predictive performance by not revealing the model's inability to predict change events. If species' distributions remain mostly stable, then even an unfit model can predict the near future well due to temporal autocorrelation. We urge caution when working with forecasts of changes in spatial patterns of species occupancy or abundance, even for SDMs that are based on time series datasets unless they are critically validated for forecasting such change.
(Less)
- author
- Piirainen, Sirke
; Lehikoinen, Aleksi
LU
; Husby, Magne
; Kålås, John Atle
; Lindström, Åke
LU
and Ovaskainen, Otso
- organization
- publishing date
- 2023
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- birds, climate change, Fennoscandia, forecasting, land use, model validation, prediction, species distribution modelling, species traits, temporal transferability
- in
- Diversity and Distributions
- volume
- 29
- issue
- 5
- pages
- 654 - 665
- publisher
- Wiley-Blackwell
- external identifiers
-
- scopus:85149691147
- ISSN
- 1366-9516
- DOI
- 10.1111/ddi.13687
- language
- English
- LU publication?
- yes
- id
- c4ce1801-df7f-4f9e-a198-7f1f4c2acf5e
- date added to LUP
- 2023-04-03 11:23:24
- date last changed
- 2024-05-16 11:13:54
@article{c4ce1801-df7f-4f9e-a198-7f1f4c2acf5e,
abstract = {{<p>Aim: Species distribution models (SDMs) are widely used to make predictions on how species distributions may change as a response to climatic change. To assess the reliability of those predictions, they need to be critically validated with respect to what they are used for. While ecologists are typically interested in how and where distributions will change, we argue that SDMs have seldom been evaluated in terms of their capacity to predict such change. Instead, typical retrospective validation methods estimate model's ability to predict to only one static time in future. Here, we apply two validation methods, one that predicts and evaluates a static pattern, while the other measures change and compare their estimates of predictive performance. Location: Fennoscandia. Methods: We applied a joint SDM to model the distributions of 120 bird species in four model validation settings. We trained models with a dataset from 1975 to 1999 and predicted species' future occurrence and abundance in two ways: for one static time period (2013–2016, ‘static validation’) and for a change between two time periods (difference between 1996–1999 and 2013–2016, ‘change validation’). We then measured predictive performance using correlation between predicted and observed values. We also related predictive performance to species traits. Results: Even though static validation method evaluated predictive performance as good, change method indicated very poor performance. Predictive performance was not strongly related to any trait. Main Conclusions: Static validation method might overestimate predictive performance by not revealing the model's inability to predict change events. If species' distributions remain mostly stable, then even an unfit model can predict the near future well due to temporal autocorrelation. We urge caution when working with forecasts of changes in spatial patterns of species occupancy or abundance, even for SDMs that are based on time series datasets unless they are critically validated for forecasting such change.</p>}},
author = {{Piirainen, Sirke and Lehikoinen, Aleksi and Husby, Magne and Kålås, John Atle and Lindström, Åke and Ovaskainen, Otso}},
issn = {{1366-9516}},
keywords = {{birds; climate change; Fennoscandia; forecasting; land use; model validation; prediction; species distribution modelling; species traits; temporal transferability}},
language = {{eng}},
number = {{5}},
pages = {{654--665}},
publisher = {{Wiley-Blackwell}},
series = {{Diversity and Distributions}},
title = {{Species distributions models may predict accurately future distributions but poorly how distributions change : A critical perspective on model validation}},
url = {{http://dx.doi.org/10.1111/ddi.13687}},
doi = {{10.1111/ddi.13687}},
volume = {{29}},
year = {{2023}},
}