From Pixels to Palate: Deep Learning-Based Image Aesthetics Assessment for Food and Beverages

• Published in: 2023 International Conference on Machine Learning and Applications (ICMLA) pp. 504 - 511
• Main Authors: Weiler, Jessica, Theissler, Andreas, Messer, Uwe
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15-12-2023
• Subjects: Computer vision; Data models; Deep learning; Feature extraction; Food and beverage photography; Image aesthetics assessment; Social networking (online); Training; Transfer learning; Visualization
• ISSN: 1946-0759
• DOI: 10.1109/ICMLA58977.2023.00076

Food and beverage images are omnipresent on the internet, reflecting a growing trend of people sharing their culi-nary experiences on various social media platforms. This visual abundance of food content presents significant opportunities and challenges for businesses and researchers alike, as they seek to harness the potential of these images' aesthetics for marketing, recommendation systems, and gaining valuable insights into consumer preferences and trends. In this paper we extend the topic's current research state (a) by composing a data set extracted from a real-world recipe data base, labeled by a large number of raters (104 laypersons) to reduce biases and enhance relevance for end-users who are typically no experts; (b) by contrasting the effects of transfer learning using pre-trained ImageNet-models compared to training from scratch and thereby investigating the usefulness of the pre- trained models' general features for aesthetics assessment; (c) by showing how the models can be used as an "Oracle" to isolate the effects of high-level image features on image aesthetics. Our experimental results show that while the test set results on our own data set are similar (approx. 80%) for pre-trained models and models trained from scratch, the pre-trained ImageNet-models have better generalization capabilities with respect to a different data set.