Advanced

Global gridded crop model evaluation : Benchmarking, skills, deficiencies and implications

Müller, Christoph; Elliott, Joshua; Chryssanthacopoulos, James; Arneth, Almut LU ; Balkovic, Juraj; Ciais, Philippe; Deryng, Delphine; Folberth, Christian; Glotter, Michael and Hoek, Steven, et al. (2017) In Geoscientific Model Development 10(4). p.1403-1422
Abstract

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time... (More)

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

(Less)
Please use this url to cite or link to this publication:
author
, et al. (More)
(Less)
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
Geoscientific Model Development
volume
10
issue
4
pages
20 pages
publisher
Copernicus Gesellschaft Mbh
external identifiers
  • scopus:85017169636
  • wos:000399100600001
ISSN
1991-959X
DOI
10.5194/gmd-10-1403-2017
language
English
LU publication?
yes
id
215787f8-b9dd-4025-8a7e-d98f02730ba7
date added to LUP
2017-05-02 08:42:56
date last changed
2017-09-18 13:33:33
@article{215787f8-b9dd-4025-8a7e-d98f02730ba7,
  abstract     = {<p>Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.</p>},
  author       = {Müller, Christoph and Elliott, Joshua and Chryssanthacopoulos, James and Arneth, Almut and Balkovic, Juraj and Ciais, Philippe and Deryng, Delphine and Folberth, Christian and Glotter, Michael and Hoek, Steven and Iizumi, Toshichika and Izaurralde, Roberto C. and Jones, Curtis and Khabarov, Nikolay and Lawrence, Peter and Liu, Wenfeng and Olin, Stefan and Pugh, Thomas Alan Miller and Ray, Deepak K. and Reddy, Ashwan and Rosenzweig, Cynthia and Ruane, Alex C. and Sakurai, Gen and Schmid, Erwin and Skalsky, Rastislav and Song, Carol X. and Wang, Xuhui and De Wit, Allard and Yang, Hong},
  issn         = {1991-959X},
  language     = {eng},
  month        = {04},
  number       = {4},
  pages        = {1403--1422},
  publisher    = {Copernicus Gesellschaft Mbh},
  series       = {Geoscientific Model Development},
  title        = {Global gridded crop model evaluation : Benchmarking, skills, deficiencies and implications},
  url          = {http://dx.doi.org/10.5194/gmd-10-1403-2017},
  volume       = {10},
  year         = {2017},
}