Multi-dimensional geospatial data mining in a distributed environment using MapReduce

• Published in: Journal of big data Vol. 6; no. 1; pp. 1 - 34
• Main Authors: Alkathiri, Mazin, Jhummarwala, Abdul, Potdar, M. B.
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 05-09-2019; Springer Nature B.V; SpringerOpen
• Subjects: Artificial intelligence; Big Data; Clustering; Communications Engineering; Computational Science and Engineering; Computer Science; Data mining; Data Mining and Knowledge Discovery; Database Management; Distributed processing; Geospatial processing; Imagery; Information Storage and Retrieval; K-means clustering; Machine learning; Mathematical Applications in Computer Science; Multi-dimensional data processing; Multiband raster processing; Networks; Raster; Spectra; Spectral bands; Spectral to geometrical space; Multiband raster processing; Geospatial processing; Spectral to geometrical space; Multi-dimensional data processing; K-means clustering
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-019-0245-9

Data mining and machine learning techniques for processing raster data consider a single spectral band of data at a time. The individual results are combined to obtain the final output. The essence of related multi-spectral information is lost when the bands are considered independently. The proposed platform is based on Apache Hadoop ecosystem and supports performing analysis on large amounts of multispectral raster data using MapReduce. A novel technique of transforming the spectral space to the geometrical space is also proposed. The technique allows to consider multiple bands coherently. The results of clustering 10
6
pixels for multiband imagery with widely used GIS software have been tested and other machine learning methods are planned to be incorporated in the platform. The platform is scalable to support tens of spectral bands. The results from our platform were found to be better and are also available faster due to application of distributed processing.