Sequential Data Processing for IMERG Satellite Rainfall Comparison and Improvement Using LSTM and ADAM Optimizer

Sequential Data Processing for IMERG Satellite Rainfall Comparison and Improvement Using LSTM and ADAM Optimizer

This study introduces a systematic methodology whereby different technologies were utilized to download, pre-process, and interactively compare the rainfall datasets from the Integrated Multi-Satellite Retrievals for Global Precipitation Mission (IMERG) satellite and rain gauges. To efficiently hand...

Full description

Saved in:
Bibliographic Details
Published in:Applied sciences Vol. 13; no. 12; p. 7237
Main Authors: Toh, Seng Choon, Lai, Sai Hin, Mirzaei, Majid, Soo, Eugene Zhen Xiang, Teo, Fang Yenn
Format: Journal Article
Language:English
Published: Basel MDPI AG 01-06-2023
Subjects:
Accuracy
ADAM
Computational linguistics
Data processing
Datasets
Deep learning
Electronic data processing
Error reduction
Gauges
Hydrology
Hypertext
IMERG satellite rainfall
Language processing
Long short-term memory
LSTM
Machine learning
Microprocessors
Natural language interfaces
Neural networks
pHp
Precipitation
Python
Rain
Rain and rainfall
Rain gauges
Rainfall
Relational data bases
Remote sensing
Root-mean-square errors
Software
SQL relational database
Time series
Wind
Malaysia
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This study introduces a systematic methodology whereby different technologies were utilized to download, pre-process, and interactively compare the rainfall datasets from the Integrated Multi-Satellite Retrievals for Global Precipitation Mission (IMERG) satellite and rain gauges. To efficiently handle the large volume of data, we developed automated shell scripts for downloading IMERG data and storing it, along with rain gauge data, in a relational database system. Hypertext pre-processor (pHp) programs were built to visualize the result for better analysis. In this study, the performance of IMERG estimations over the east coast of Peninsular Malaysia for the duration of 10 years (2011–2020) against rain gauge observation data is evaluated. Moreover, this study aimed to improve the daily IMERG estimations with long short-term memory (LSTM) developed with Python. Findings show that the LSTM with Adaptive Moment Estimation (ADAM) optimizer trained against the mean square error (MSE) loss enhances the accuracy of satellite estimations. At the point-to-pixel scale, the correlation between satellite estimations and ground observations was increased by about 15%. The bias was reduced by 81–118%, MAE was reduced by 18–59%, the root-mean-square error (RMSE) was reduced by 1–66%, and the Kling–Gupta efficiency (KGE) was increased by approximately 200%. The approach developed in this study establishes a comprehensive and scalable data processing and analysis pipeline that can be applied to diverse datasets and regions encountering similar domain-specific challenges.
ISSN:2076-3417
2076-3417
DOI:10.3390/app13127237