Classifying the content of social media images to support cultural ecosystem service assessments using deep learning models

• Published in: Ecosystem services Vol. 54; p. 101410
• Main Authors: Cardoso, Ana Sofia, Renna, Francesco, Moreno-Llorca, Ricardo, Alcaraz-Segura, Domingo, Tabik, Siham, Ladle, Richard J., Vaz, Ana Sofia
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-04-2022
• Subjects: Computer vision; Convolutional neural networks; Culturomics; iEcology; Nature contributions to people; Transfer learning; Nature contributions to people; Computer vision; Convolutional neural networks; iEcology; Transfer learning; Culturomics
• ISSN: 2212-0416; 2212-0416
• DOI: 10.1016/j.ecoser.2022.101410

•We used freely available deep learning models to analyse social media content in two distinct Iberian areas.•Our models allowed accurate classification of nature attributes underlying cultural ecosystem services (CES)•In Peneda-Gerês (Portugal) results showed preference for living systems supporting aesthetics appreciation.•Conversely, elements resonant in terms of culture and heritage were preferred in Sierra-Nevada (Spain).•We demonstrate that deep learning can be used for assessing CES-relevant content from crowdsourced data.
Crowdsourced social media data has become popular for assessing cultural ecosystem services (CES). Nevertheless, social media data analyses in the context of CES can be time consuming and costly, particularly when based on the manual classification of images or texts shared by people. The potential of deep learning for automating the analysis of crowdsourced social media content is still being explored in CES research. Here, we use freely available deep learning models, i.e., Convolutional Neural Networks, for automating the classification of natural and human (e.g., species and human structures) elements relevant to CES from Flickr and Wikiloc images. Our approach is developed for Peneda-Gerês (Portugal) and then applied to Sierra Nevada (Spain). For Peneda-Gerês, image classification showed promising results (F1-score ca. 80%), highlighting a preference for aesthetics appreciation by social media users. In Sierra Nevada, even though model performance decreased, it was still satisfactory (F1-score ca. 60%), indicating a predominance of people’s pursuit for cultural heritage and spiritual enrichment. Our study shows great potential from deep learning to assist in the automated classification of human-nature interactions and elements from social media content and, by extension, for supporting researchers and stakeholders to decode CES distributions, benefits, and values.