Food Ingredients Recognition through Multi-label Learning Marc - PowerPoint PPT Presentation

Food Ingredients Recognition through Multi-label Learning Marc Bolaños, Aina Ferrà and Petia Radeva

Motivation

Motivation INPUT OUTPUT Computer List of Nutritional Vision Ingredients Composition Algorithm

dishes ingredients Related Work Only one work has been proposed for ingredients recognition. Jingjing Chen and Chong-Wah Ngo. “Deep-based ingredient recognition for cooking recipe retrieval”. In Proceedings of the 2016 ACM on Multimedia Conference, pages 32–41. ACM, 2016.

dishes ingredients Related Work Only one work has been proposed for ingredients recognition. Handicaps of their proposal: - Both dish and ingredients information is needed. Not applicable for never-seen recipes. - Applicable to visible ingredients only. Jingjing Chen and Chong-Wah Ngo. “Deep-based ingredient recognition for cooking recipe retrieval”. In Proceedings of the 2016 ACM on Multimedia Conference, pages 32–41. ACM, 2016.

Conventional CNN for classification Softmax loss Categorical cross-entropy egg

Our proposal: Adaptation for Multi-label recognition Sigmoid mustard loss salt Binary cross-entropy paprika egg

Datasets - Ingredients101 Carrot Cake Baby Back Ribs Dataset complementary to Food101: - 101 classes / dishes - 1000 images per class A recipe for each class was downloaded from resulting in a list of ingredients per class and a total of 446 unique ingredients.

Datasets - Recipes5k New dataset acquired from Around 50 different recipes were downloaded for each class in Food101 together with their respective picture. Resulting in 4,826 images, a list of ingredients per image and a total of 3,213 unique ingredients.

Datasets - Recipes5k Ingredients Simplification Two problems arise when dealing with too many labels: - The amount of training samples needed for learning them. - The ambiguity and minor differences between them. We propose a simplified version by applying a simple removal of overly-descriptive particles (e.g. ’sliced’ or ’sauce’). Simplifying the 3,213 ingredients into 1,013. large eggs → egg lemon zest → lemon meyer lemon juice → lemon unsalted butter → butter boiling water → water

Results - Ingredients101

Results - Recipes5k INGREDIENTS SIMPLIFIED

Results - Recipes5k

Neurons’ Activations

Conclusions We have proposed: - Model suitable for ingredients recognition through multi-label learning. - Two datasets for ingredients recognition benchmarking. Advantages of our proposal with respect to the state of the art: - Straightforward model applicable to any highly performing CNN. - Dish/class information is not used for learning. Implying that the ingredients can be inferred from never-seen dishes. - Can directly learn ingredients’ representation from visual appearance. - Can predict invisible ingredients implicitly.

THANK YOU FOR YOUR ATTENTION www.ub.edu/cvub/marcbolanos marc.bolanos@ub.edu

Food Ingredients Recognition through Multi-label Learning Marc - PowerPoint PPT Presentation

Food Ingredients Recognition through Multi-label Learning Marc Bolaos, Aina Ferr and Petia Radeva Motivation Motivation Motivation INPUT OUTPUT Computer List of Nutritional Vision Ingredients Composition Algorithm dishes

Multi-label Learning Approaches for Music Instrument Recognition Eleftherios Spyromitros - Xioufis

Learning Context-dependent Label Permutations for Multi-label Classification Jinseok Nam Amazon

Work on Multi-label Classification Jesse Read Supervised by Bernhard Pfahringer

The most important trend for in the field of food and beverages is: Clean Label Consumers want

Drawing Parallels between Multi-label Classification and Multi-target Regression Grigorios

DEVEL DEVELOPMENT OF OPMENT OF INDONESIAN FOOD INDONESIAN FOOD INGREDIENTS INDUSTR INGREDIENTS

Factorization of the Label Conditional Distribution for Multi-Label Classification ECML PKDD 2015

Multi-label Classification Charmgil Hong cs3750 (Presented on Nov 11, 2014) Goals of the talk

Multi-Label Learning with Highly Incomplete Data via Collaborative Embedding Yufei Han 1 , Guolei

Food Image Recognition Using Very Deep Convolutional Networks Hamid Hassannejad 2 nd

SIGML Department of CSE IIT Kanpur Special Interest Group in Machine Learning Classification

Labeling Information Enhancement for Multi-label Learning with Low-rank Subspace An Tao* , Ning

Multiclass and Multi-label Classification INFO-4604, Applied Machine Learning University of

On-line Hierarchical Multi-label Text Classification Jesse Read September 7, 2007 On-line

Philosophy Energy Food Water Food Freshwater or Saltwater? BOTH 17 Different Ingredients to

Titanium dioxide (E 171): update on EFSAs activities Federica Lodi Food Ingredients and

Classify as a Whole? MULTIPLE INSTANCE LEARNING Set Learning? Multi-Set Learning? Marco Loog

Efficient Multi-Instance Learning for Activity Recognition from Time Series Data Using an

Multi-Task Transfer Learning for Fine-Grained Named Entity Recognition Masato Hagiwara 1 , Ryuji

Uncertainty-Aware Food Recognition by Deep Learning Petia Radeva, Collaboration with: Eduardo

All Natural Ingredients Arctic ECO Green is made from all natural wholesome ingredients

On-line Hierarchical Multi-label Text Classification Jesse Read Supervised by Bernhard (and Eibe

1 Explanation of Off-Label Use 1 Off-label prescribing refers to use that is not included in

Fast Direct Search in an Optimally Compressed Continuous Target Space for Efficient Multi-Label