Multitask Convolutional Neural Network Emulators for Global Crop Models - Supervised Deep Learning in Large Hypercubes of Non-IID Data

Abstract

The aim of this thesis is to establish whether a neural network (NN) can be used for emulation of simulated global crop production - retrieved from the computationally demanding dynamic global vegetation model (DGVM) Lund-Potsdam-Jena General Ecosystem Simulator (LPJ-GUESS). It has been devoted to elaboration with various types of neural network architectures: Branched NNs capable of processing inputs of mixed data types; Convolutional Neural Network (CNN) architectures able to perform automated temporal feature extraction of the given weather time series; simpler fully connected (FC) structures as well as Multitask NNs. The NN crop emulation approach was tested for approximation of global annual spring wheat responses to changes in carbon dioxide, temperature, precipitation, and nitrogen fertilizer levels.

The model domain is a multidimensional hyperspace of non-IID samples that can be blocked into climate-, temporal- and global position factor levels. NNs tend to focus on the normally behaving samples and take less notice of rare behaviors and relations. In this vast data set, the varying characteristics and relations can thus be hard to detect - even for a large and complex neural network. Due to these complexities in the LPJ-GUESS sample distributions and because neural network learning is heavily reliant on the data, a fair share of the thesis has been dedicated to network training and sample selection - with the purpose of improving learning without causing the network to overfit.

Further, the Köppen climate classification system, based on historical climate and vegetation, was used for aggregation of the emulator domain - in order to form smaller homogeneous groups. It is easier to dissect the data when these disjoint groups are analysed in isolation, which in turn can facilitate input variable selection. Moreover, by aggregating the model domain and allowing for separate deep learning of each domain-fraction, several sub-models can be constructed and trained for a specific Köppen climate region. These can then be combined into an integrated composite emulator. In contrast to an emulator trained to model crop production for the whole domain, a model composition emulator does not have to account for the differences between the sub-domains and hence only has to focus on learning the within-group relations and patterns in the disjoint climate classes.

Popular Abstract

A growing world population alongside climate change and greater uncertainties in weather increases the concern about food security and raises questions about vulnerabilities and potential adaptation strategies in the agricultural sector. Dynamic Global Vegetation Models (DGVM) can be used for forecasting of vegetation, both in changing climate settings, out of which many have not yet been seen in the historical record, as well as at all potential cultivation locations, including those that have no previous record of food production. The problem is that DGVMs often are very large and complex systems, making the simulations computationally demanding. This has eventuated in the need of much simpler emulators that by mimicking the behavior of the DGVM, can produce estimated predictions at a lower computational cost.

The aim of this thesis is to establish whether a neural network (NN) model can be used for emulation, a.k.a surrogate modeling, of simulated global crop production, retrieved from the computationally demanding DGVM Lund-Potsdam-Jena General Ecosystem Simulator (LPJ-GUESS).

Neural networks are heavily reliant on the data and tend to focus on the normally behaving samples that make up the majority of the data used for learning the relations between the simulator inputs and outputs. Thence NNs takes less notice of rare behaviors and relations, out of which some can provide useful information. The diversity of the global crop responses to shifts in climate - caused by variation in the impact of the different climate scenarios at different geographical locations, among which many diverge from the mainstream - can thus aggravate the learning of general behaviors. For example, an increase in temperature can have a huge impact on crop production at some locations and hardly any effect at others.

By separating heterogeneous data samples through aggregation of the model domain and letting these disjoint sample groups be modeled individually, separate emulators can focus on finding features of importance within those sub-groups and then be combined into an integrated composite emulator. In this thesis, the globe was divided into separate regions according to the widely used Köppen climate classification system - derived from historical climate and vegetation. I.e a model composition was used for emulation and the sub-models were individually trained to model crop responses to climate change in these different Köppen climate regions.

Further, surrogate modeling often involves some sample selection strategy, determining which samples the emulator should learn from. In this thesis, such handling of samples, through aggregation and sample selection, left room for improvement, but also proved to facilitate the learning of as many samples as possible, without losing the ability to predict for yet unseen scenarios and location.

• author: Nilsson, Amanda
• supervisor: Johan Lindström
• organization: Mathematical Statistics
• course: MASM01 20192
• year: 2020
• type: H2 - Master's Degree (Two Years)
• subject: Mathematics and Statistics
• keywords: Multitask Learning, Convolutional Neural Network (CNN), Branched Neural Network, Dynamic Global Vegetation Models (DGVM), Automated Feature Extraction, Feature Importance, Supervised Machine Learning, Emulator, Surrogate Model, Response Surface Model, Approximation Model, Metamodeling, Model Composition, Regularization, Robustness, Hyperparameter Optimization
• publication/series: Master's Theses in Mathematical Sciences
• report number: LUNFMS-3093-2020
• ISSN: 1404-6342
• other publication id: 2020:E59
• language: English
• id: 9023045
• date added to LUP: 2020-09-07 14:51:37
• date last changed: 2021-06-04 18:17:22