Crop type classification using multi-sensor satellite image time series data and attention-based deep learning technique: a case study of southern India

Abstract

Crop type classification is essential for effective agricultural monitoring in India. Preforming crop type classification in the diverse Indian agriculture setting is a challenging task. This thesis leverages transformer-based deep learning model, specifically the Pixel-Set Encoder Temporal Attention Encoder (PSETAE), to classify ten crop types of rabi season (2024 - 25) using Satellite Image Time Series (SITS) data. The thesis provides an approach that integrates mobile-based field surveys, Google Earth Engine (GEE) based data extraction and preprocessing, and model implementation using PyTorch. Field data involving polygons with the crop label were collected from two regions of Karnataka, India, using ArcGIS Field Maps. The polygons served as the basis for extracting SITS data, which were then utilized for crop classification. Multiple SITS data - Sentinel-1, Sentinel-2, and Harmonized Landsat Sentinel-2 datasets (HLSS30 and HLSL30) - and data fusion methods were evaluated using PSETAE crop type classification model. The PSETAE model achieved high classification accuracies for Sentinel-2, followed by HLSS30 and HLSL30 and Sentinel-1 achieved least. Fusion Methods achieved accuracy between 94-95 % which is slightly less than Sentinel-2. Spatial generalization tests revealed a remarkable inversion of results suggesting that Synthetic Aperture Radar (SAR) data from Sentinel-1 captures more transferable information about crops than Sentinel-2. Both HLSS30 and HLSL30 performed well in the test area but failed to maintain their performance during generalization. While overall generalization accuracy was low, strong F1 scores for chickpea (0.92), wheat (0.80), and sugarcane (0.79) from Sentinel-1, and fruit plantations from HLSL30 (0.82) and Statistical Average Fusion (0.83), highlight the potential of Sentinel-1, data fusion methods and model ensembling to enhance operational crop type classification.

Popular Abstract

This study focused on creating crop maps using satellite images and machine learning. The goal was to develop crop type map by combining mobile surveys, cloud-based data processing, and machine learning in one smooth process.
The study was done in Vijayapura, a district in Karnataka, India, during the Rabi season (September to March 2024–25). For the first time in this region, over 4,000 fields were surveyed using a mobile app called Esri’s Field Maps. The collected field polygons were used for satellite data extraction. We used Google Earth Engine (GEE), a powerful cloud platform, to process satellite images quickly and efficiently. These included satellite images from Sentinel-1, Sentinel-2, and Harmonized Landsat & Sentinel-2 (HLS) satellites. To identify crop types, we used machine learning models built in Python, a popular programming language for data science. A special type of machine learning model called a transformer helped us analyze the satellite data and make accurate predictions.
The results were impressive. The model using Sentinel-2 data gave the best accuracy—up to 96%. Sentinel-1 gave the lowest accuracy, while HLS was somewhere in between. However, during cloudy months (September to December), combining data from different satellites (called data fusion) worked better than using any single satellite alone. To see if the model works in other places, we also tested it on the data collected from Dharwad, a nearby district. Interestingly, in that area, the models using Sentinel-1 and fusion data performed better than those using Sentinel-2.
This study highlights the importance of adapting crop classification approaches to the specific characteristics of each region. Rather than relying on a single satellite dataset, integrating multiple sources - especially through fusion methods - can provide more stable and reliable outcomes. Approaches that combine optical and radar data offer advantages in handling variable conditions like cloud cover or geographic diversity.
Overall, this project shows that combining mobile surveys, satellite images, cloud computing, and machine learning can produce highly accurate crop maps. This method can help farmers, researchers, and policymakers make better decisions in agriculture.

• author: Yaragal, Shivaprakash
• supervisor: Ali Mansourian; Rachid Oucheikh
• organization: Dept of Physical Geography and Ecosystem Science
• alternative title: Grödklassificering med hjälp av multisen satellitbildstidsseriedata och uppmärksamhetsbaserad djupinlärningsteknik: En fallstudie från södra Indien
• course: NGEM01 20251
• year: 2025
• type: H2 - Master's Degree (Two Years)
• subject: Earth and Environmental Sciences
• keywords: agriculture, crop type classification, transformer-based deep learning, PSETAE, SITS, Sentinel-1, Sentinel-2, HLS products, fusion methods, ArcGIS, GEE, PyTorch
• publication/series: Student thesis series INES
• report number: 748
• funder: Crafoord Foundation
• language: English
• id: 9204066
• date added to LUP: 2025-06-23 13:30:05
• date last changed: 2025-06-23 13:30:05