Earth Observation Data for Machine Learning: A Comprehensive Approach for Collecting, Preprocessing, and Integrating Data Sets
- Paper ID
77115
- author
- company
BRAC University
- country
Bangladesh
- year
2023
- abstract
Big Data, cloud-based operations, Internet of Things and crowdsourcing are the recently developed information and web-based technologies. However finding data sets for these Earth observation (EO) data processing systems requires tremendous labor. In addition data acquisition, communication, processing, dissemination, and archiving seems to be difficult and time-consuming as classified datasets are difficult to find. Some novel approaches must be found for taking preparation for data analysis, extracting information from large data sets, and promptly making the information much more accessible to researchers. We developed a method for processing EO data for machine learning that includes cutting-edge technology with the infrastructure of EO data processing . The website we created offers tools like in-depth search choices, data filtering, and data previewing to help users identify and select the most relevant and superior data for their use in research projects. In order to facilitate the researcher's work, we have created a Python script and a Chrome extension that allow users to search for and download data sets according to certain criteria. Despite these there are a tremendous number of data that gets corrupted and becomes futile because of the image quality, focus, solar flare radiation and cloud coverage; welcoming these data alongside the clear perfect images occupies ample space and prevent sending useful images to earth station for analysis as there is an onboard space shortage for storing. To solve this problem we developed an algorithm latched with ML which not only is capable of comparing effectively but also sends flawless images by excluding the faulty images automatically consequently saving time as well as storage and boosting the efficiency of the research process. In a nutshell, our methodology provides a modern solution to the elderly problem of handling EO data where we use machine learning in the context of developing information and web-based technology. In addition, we can manage EO data processing systems and connect the general public to these applications much more effectively by utilizing contemporary technology and comprehending their disruptive potential. Our approach with data collecting, integration, and distribution makes it a useful tool for researchers, practitioners, and decision-makers in environmental and catastrophe management as it is scalable and customizable while finding datasets for Machine Learning applications particularly for Earth Observation.