amecon abstract meta concept features for text
play

AMECON: Abstract Meta-Concept Features for Text Illustration Ines - PowerPoint PPT Presentation

Oct 22, 2022 •316 likes •762 views

AMECON: Abstract Meta-Concept Features for Text Illustration Ines Chami 1, *, Youssef Tamaazousti 2, * and Herv Le Borgne 2 1: Stanford University, USA 2: CEA LIST, FRANCE * Both authors contributed equally | 1 ICMR 2017 | Tamaazousti

  1. AMECON: Abstract Meta-Concept Features for Text Illustration Ines Chami 1, *, Youssef Tamaazousti 2, * and Hervé Le Borgne 2 1: Stanford University, USA – 2: CEA LIST, FRANCE * Both authors contributed equally | 1 ICMR 2017 | Tamaazousti Youssef

  2. Text-illustration System • SVM one-vs-all • Multi-Layer neural network • etc. | 2 ICMR 2017 | Tamaazousti Youssef

  3. Cross-Modal Retrieval Task • Cross-Modal Retrieval task • Given a document in one modality, find (from database) the most relevant documents in another modality • Text-illustration • SVM one-vs-all • Multi-Layer neural network • Query: sentences • etc. • Collection: images • Hard problem: semantic gap | 3 ICMR 2017 | Tamaazousti Youssef

  4. Cross-Modal Retrieval Approach 1 • Canonical Correlation Analysis • Hardoon et al. Neural Computation 2004 • Hwang and Grauman, IJCV 2012 • Costa Pereira et al. TPAMI 2014 • Tran et al., CVPR 2016 • etc. • SVM one-vs-all • Multi-Layer neural network • etc. | 4 ICMR 2017 | Tamaazousti Youssef

  5. Cross-Modal Retrieval Approach 2 • Neural Network (NN) • Karpathy and Fei-Fei, NIPS 2014 • Yan and Mikolajczyk, CVPR 2015 • Karpathy and Fei-Fei, CVPR 2015 • Mao et al., ICLR 2015 • Kiros et al., TACL 2015 • Wang et al., CVPR 2016 • etc. | 5 ICMR 2017 | Tamaazousti Youssef

  6. Main Principle of NN Approach | 6

  7. Main Principle of NN Approach | 7

  8. Main Principle of NN Approach | 8

  9. Main Principle of NN Approach | 9

  10. This work: New Approach | 10 ICMR 2017 | Tamaazousti Youssef

  11. This work: New Approach | 11 ICMR 2017 | Tamaazousti Youssef

  12. This work: New Approach | 12 ICMR 2017 | Tamaazousti Youssef

  13. This work: New Approach | 13 ICMR 2017 | Tamaazousti Youssef

  14. AMECON principle • AMECON: Abstract Meta-CONcept • Abstract-concept + Meta-concept • SVM one-vs-all • Multi-Layer neural network • etc. | 14 ICMR 2017 | Tamaazousti Youssef

  15. AMECON principle • AMECON: Abstract Meta-CONcept • Abstract-concept + Meta-concept • SVM one-vs-all • Multi-Layer neural network • etc. | 15 ICMR 2017 | Tamaazousti Youssef

  16. AMECON principle • AMECON: Abstract Meta-CONcept • Abstract-concept + Meta-concept • SVM one-vs-all • Multi-Layer neural network • etc. | 16 ICMR 2017 | Tamaazousti Youssef

  17. AMECON principle • AMECON: Abstract Meta-CONcept • Abstract-concept + Meta-concept • SVM one-vs-all • Multi-Layer neural network • etc. | 17 ICMR 2017 | Tamaazousti Youssef

  18. AMECON principle • AMECON: Abstract Meta-CONcept • Abstract-concept + Meta-concept • SVM one-vs-all • Multi-Layer neural network • etc. | 18 ICMR 2017 | Tamaazousti Youssef

  19. Overview of Our Approach | 19 ICMR 2017 | Tamaazousti Youssef

  20. Learning Text2Amecon block • Learning Textual Features a. Select all different words from training-data b. Remove stop-words (``is’’, ``of’’, ``for’’, etc.) c. Compute word2vec features for each word d. Cluster (k-means) the whole set of features | 20 ICMR 2017 | Tamaazousti Youssef

  21. Learning Text2Amecon block • Learning Textual Features a. Select all different words from training-data b. Remove stop-words (``is’’, ``of’’, ``for’’, etc.) c. Compute word2vec features for each word d. Cluster (k-means) the whole set of features AMECONs | 21 ICMR 2017 | Tamaazousti Youssef

  22. Computing Textual AMECON Features Test phase • SVM one-vs-all • Multi-Layer neural network • etc. | 22 ICMR 2017 | Tamaazousti Youssef

  23. Overview of Our Approach | 23 ICMR 2017 | Tamaazousti Youssef

  24. Learning Image2Amecon block | 24 ICMR 2017 | Tamaazousti Youssef

  25. Learning Image2Amecon block | 25 ICMR 2017 | Tamaazousti Youssef

  26. Learning Image2Amecon block | 26 ICMR 2017 | Tamaazousti Youssef

  27. Learning Image2Amecon block | 27 ICMR 2017 | Tamaazousti Youssef

  28. Learning Image2Amecon block | 28 ICMR 2017 | Tamaazousti Youssef

  29. Learning Image2Amecon block | 29 ICMR 2017 | Tamaazousti Youssef

  30. Computing Visual AMECON Features Test phase • SVM one-vs-all • Multi-Layer neural network • etc. | 30 ICMR 2017 | Tamaazousti Youssef

  31. Overview of Our Approach | 31 ICMR 2017 | Tamaazousti Youssef

  32. Matching Multi-Modal Data in AMECON Space • Matching texts & images in the same AMECON Space • Text and Images directly comparable • Perform ANY multi-modal task • SVM one-vs-all • Multi-Layer neural network • etc. | 32 ICMR 2017 | Tamaazousti Youssef

  33. Text-Illustration in AMECON Space • SVM one-vs-all • Multi-Layer neural network • etc. | 33 ICMR 2017 | Tamaazousti Youssef

  34. Text-Illustration in AMECON Space • SVM one-vs-all • Multi-Layer neural network • etc. | 34 ICMR 2017 | Tamaazousti Youssef

  35. Text-Illustration in AMECON Space • SVM one-vs-all • Multi-Layer neural network • etc. | 35 ICMR 2017 | Tamaazousti Youssef

  36. Text-Illustration in AMECON Space • SVM one-vs-all • Multi-Layer neural network • etc. | 36 ICMR 2017 | Tamaazousti Youssef

  37. Experimental Protocol • Training data •6,000/30,000 images in Flickr-8k/Flickr-30k •Each image associated to 5 captions • Testing data (same for Flickr-8k & 30k) •1000 images and 5000 captions • SVM one-vs-all • Multi-Layer neural network •All captions as data-queries • etc. •All images as data-collection •Evaluation metric: Recall@K (K = 1, 5, 10) | 37 ICMR 2017 | Tamaazousti Youssef

  38. Text Illustration Results Neural Network-based Approach CCA-based Approach Our Approach | 38 ICMR 2017 | Tamaazousti Youssef

  39. Text Illustration Results Neural Network-based Approach CCA-based Approach Our Approach | 39 ICMR 2017 | Tamaazousti Youssef

  40. Text Illustration Results Neural Network-based Approach CCA-based Approach Our Approach | 40 ICMR 2017 | Tamaazousti Youssef

  41. Text Illustration Results Neural Network-based Approach CCA-based Approach Our Approach | 41 ICMR 2017 | Tamaazousti Youssef

  42. Analysis of Parameters • Quite robust to the parameters •Robust to #selected neighbours •Sensitive to #clusters (C) but stable when for a large range of C values • SVM one-vs-all • Multi-Layer neural network • etc. | 42 ICMR 2017 | Tamaazousti Youssef

  43. Conclusion • Novelty: • Principle of AMECONs • Abstract MEta-CONcepts • Mixing supervised and unsupervised learning to build a multi-modal space • Results on Text-illustration: • SVM one-vs-all • +4 points of R@K (avg.) compared to best • Multi-Layer neural network • etc. methods of the literature • Future Work: • Image captioning with AMECON-features | 43 ICMR 2017 | Tamaazousti Youssef

  44. Code will be released at: http://perso.ecp.fr/~tamaazouy/ Thank you (questions ?) Commissariat à l’énergie atomique et aux énergies alternatives Institut List | CEA SACLAY NANO-INNOV | BAT. 861 – PC142 91191 Gif-sur-Yvette Cedex - FRANCE www-list.cea.fr Établissement public à caractère industriel et commercial | RCS Paris B 775 685 019 | 44

Recommend

Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. Rob van der Goot, Nikola

Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. Rob van der Goot, Nikola

Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. Rob van der Goot, Nikola Ljube si c, Ian Matroos, Malvina Nissim & Barbara Plank Bleaching Text: Abstract Features for Cross-lingual Gender Prediction. Rob van

791 views • 41 slides
AT ATI TEAS READING REVIEW PART 4 UNDERSTANDING TEXT FEATURES and REFERENCE SOURCES Text

AT ATI TEAS READING REVIEW PART 4 UNDERSTANDING TEXT FEATURES and REFERENCE SOURCES Text

AT ATI TEAS READING REVIEW PART 4 UNDERSTANDING TEXT FEATURES and REFERENCE SOURCES Text features are defined as the elements that stand out in a text because the passage uses a tool to emphasize them. Several purposes for text features include

391 views • 3 slides
Considered ELA Features Sunday, July 15, 2018 9 / 20 Considered ELA Features Meta-Model

Considered ELA Features Sunday, July 15, 2018 9 / 20 Considered ELA Features Meta-Model

Considered ELA Features Sunday, July 15, 2018 9 / 20 Considered ELA Features Meta-Model Features: fits linear and quadratic models (with and without pairwise interaction e ff ects) to the data extracts information from these models, such as

436 views • 7 slides
24) Exchange Syntax and Textual 3.Advanced features 1. Mapping text to data types DSLs using

24) Exchange Syntax and Textual 3.Advanced features 1. Mapping text to data types DSLs using

Outline 1.Introduction 2.How to build a DSL 1. Defining/Using a meta model 2. Syntax Definition 1. Generating an initial syntax (HUTN) 3. Refining the syntax 24) Exchange Syntax and Textual 3.Advanced features 1. Mapping text to data types

435 views • 14 slides
The MEDEORA-HIV Database Agenda Concept Concept Features Interfaces Concept Quality

The MEDEORA-HIV Database Agenda Concept Concept Features Interfaces Concept Quality

The MEDEORA-HIV Database Agenda Concept Concept Features Interfaces Concept Quality Assurance Reporting Development of a Database for local use in outpatient centers AREVIR Standardized software and data format Future

448 views • 17 slides
Concept Detection: Concept Detection: Convergence to Local Features Convergence to Local Features

Concept Detection: Concept Detection: Convergence to Local Features Convergence to Local Features

Concept Detection: Concept Detection: Convergence to Local Features Convergence to Local Features and Opportunities Beyond and Opportunities Beyond Shih Fu Chang 1 , Junfeng He 1 , Yu Gang Jiang 1,2 , Elie El Khoury 3 , Chong Wah Ngo 2

604 views • 33 slides
Concept and text by

Concept and text by

Mozan as Urkesh: Archaeology in the Making Concept and text by Giorgio Buccellati Arabic translation by Rasha Endari Lojain Hatahet Yasmine Mahmud 2008 B y definition,

551 views • 24 slides
Abstract Meta-learning, or learning to learn, has gained renewed interest in recent years within

Abstract Meta-learning, or learning to learn, has gained renewed interest in recent years within

Abstract Meta-learning, or learning to learn, has gained renewed interest in recent years within the artificial intelligence community. However, meta-learning is incredibly prevalent within nature, has deep roots in cognitive science and

864 views • 74 slides
SML CONCEPT 1 SML CONCEPT 1 SML CONCEPT 1 Design 1 Main features 1 Large yoke arms and yoke

SML CONCEPT 1 SML CONCEPT 1 SML CONCEPT 1 Design 1 Main features 1 Large yoke arms and yoke

SML CONCEPT 1 SML CONCEPT 1 SML CONCEPT 1 Design 1 Main features 1 Large yoke arms and yoke base 2 Deep cast aluminium pan tube base 3 Pan tube base can incorporate components as well as display and keypad 4 Separate double ended handle

509 views • 38 slides
reliable innovation Main features Main features Lower part detachment Concept advantages

reliable innovation Main features Main features Lower part detachment Concept advantages

SIRE-NN Chocolate enrobing reliable innovation Main features Main features Lower part detachment Concept advantages Cantilever design for easy access (cleaning and maintenance) Detail features Detail features Pilot plant Compact

600 views • 13 slides
Learning to generate: Concept-to-text generation using machine learning Ioannis Konstas

Learning to generate: Concept-to-text generation using machine learning Ioannis Konstas

Learning to generate: Concept-to-text generation using machine learning Ioannis Konstas Institute for Language, Cognition and Computation University of Edinburgh Aberdeen, NLG Summer School 21 July 2015 Konstas (ILCC) Concept-to-Text

1.83k views • 143 slides
Inferring Meta-Data Patricia Helmich Basiert auf dem Paper: Tandeep Sidhu; Judith Klavans; Jimmy

Inferring Meta-Data Patricia Helmich Basiert auf dem Paper: Tandeep Sidhu; Judith Klavans; Jimmy

Projektseminar: Text Mining for Historical Documents (WS 2010/11) Inferring Meta-Data Patricia Helmich Basiert auf dem Paper: Tandeep Sidhu; Judith Klavans; Jimmy Lin. Concept Disambiguation for Improved Subject Access Using Multiple Knowledge

377 views • 13 slides
Unsupervised Concept-to-text Generation with Hypergraphs Ioannis Konstas, Mirella Lapata

Unsupervised Concept-to-text Generation with Hypergraphs Ioannis Konstas, Mirella Lapata

Unsupervised Concept-to-text Generation with Hypergraphs Ioannis Konstas, Mirella Lapata Institute for Language, Cognition and Computation University of Edinburgh NAACL 2012, Montral Konstas, Lapata (ILCC) Unsupervised Concept-to-text

1.47k views • 62 slides
Lecture 22: Meta-Modelling Inheritance Meta-Modelling Idea Experiment : can we model

Lecture 22: Meta-Modelling Inheritance Meta-Modelling Idea Experiment : can we model

Content Inheritance Abstract syntax Software Design, Modelling and Analysis in UML Liskov Substitution Principle Well-typedness with inheritance Subset-semantics vs. uplink-semantics Lecture 22: Meta-Modelling Inheritance

749 views • 12 slides
Meta Queries Workshop Scott Joyce Advanced Meta Queries Which table do I use? How do I

Meta Queries Workshop Scott Joyce Advanced Meta Queries Which table do I use? How do I

Meta Queries Workshop Scott Joyce Advanced Meta Queries Which table do I use? How do I link two or more tables? Tips and Tricks New Features 2 Reference Guides & Standard Tables Keystone has delivered several standard

603 views • 44 slides
Speech Generation From Concept and from Text Martin Jansche CS 6998 2004-02-11 Components of

Speech Generation From Concept and from Text Martin Jansche CS 6998 2004-02-11 Components of

Speech Generation From Concept and from Text Martin Jansche CS 6998 2004-02-11 Components of spoken output systems Front end: From input to control parameters. From naturally occurring text; or From constrained mark-up language; or

420 views • 27 slides
Automated sleep scoring using unsupervised learning of meta-features DD221X: Degree project in

Automated sleep scoring using unsupervised learning of meta-features DD221X: Degree project in

Automated sleep scoring using unsupervised learning of meta-features DD221X: Degree project in Computer Science May 3rd 2016 Sebastian Olsson What is sleep scoring? Judgments about a sleeping individual Sleep stages: REM, N1, N2,

662 views • 36 slides
CSC 1800 Organization of Programming Languages Abstract Data Types 1 The Concept of

CSC 1800 Organization of Programming Languages Abstract Data Types 1 The Concept of

CSC 1800 Organization of Programming Languages Abstract Data Types 1 The Concept of Abstraction An abstraction is a view or representation of an entity that includes only the most significant attributes The concept of abstraction is

538 views • 14 slides
30 Years of Abstract Interpretation Thomas Jaudon Ball Microsoft Research [This is the text of a

30 Years of Abstract Interpretation Thomas Jaudon Ball Microsoft Research [This is the text of a

30 Years of Abstract Interpretation Thomas Jaudon Ball Microsoft Research [This is the text of a 20 minute talk I gave at the 30 Years of Abstract Interpretation Workshop in San Francisco, California, USA on January 9, 2008. The subsections

196 views • 9 slides
Authorship Identification with Modality Specific Meta Features Thamar Solorio, Sangita Pillay,

Authorship Identification with Modality Specific Meta Features Thamar Solorio, Sangita Pillay,

Authorship Identification with Modality Specific Meta Features Thamar Solorio, Sangita Pillay, Manuel Montes, Natural Language Processing Lab University of Alabama at Birmingham Thamar Solorio (UAB) PAN 2011 1 / 11 Introduction Introduction

549 views • 11 slides
The Meta-structure of Knowledge and the Explanatory Gap Object, Time, Concept, Meaning, Reference

The Meta-structure of Knowledge and the Explanatory Gap Object, Time, Concept, Meaning, Reference

The Meta-structure of Knowledge and the Explanatory Gap Object, Time, Concept, Meaning, Reference and Perception Jos M. Matas jmmatias@uvigo.es TSC, Stockholm May 3th, 2011 http://webs.uvigo.es/jmmatias/knowledge/knowledge.htm Outline

505 views • 21 slides
WEBINAR 30 th APRIL 2020 Maysir UNIQUE FEATURES OF TAKAFUL COMPARED TO INSURANCE Concept of

WEBINAR 30 th APRIL 2020 Maysir UNIQUE FEATURES OF TAKAFUL COMPARED TO INSURANCE Concept of

WEBINAR 30 th APRIL 2020 Maysir UNIQUE FEATURES OF TAKAFUL COMPARED TO INSURANCE Concept of profit Based on sharing Islamic Principles Mudarabah Unique Features of Takaful Kafala ( jointly Participants guarantee one (Customers) are

114 views • 9 slides
Abstract Linear Algebra , my undergraduate text, has long been freely available online. I

Abstract Linear Algebra , my undergraduate text, has long been freely available online. I

Abstract Linear Algebra , my undergraduate text, has long been freely available online. I recently adapted it for paper. I will discuss some differences in delivering work in the two formats that current authors can apply, and some things about

604 views • 49 slides
CSc 337 LECTURE 28: SESSIONS AND WRAPPING UP What is a session? session : an abstract concept

CSc 337 LECTURE 28: SESSIONS AND WRAPPING UP What is a session? session : an abstract concept

CSc 337 LECTURE 28: SESSIONS AND WRAPPING UP What is a session? session : an abstract concept to represent a series of HTTP requests and responses between a specific Web browser and server HTTP doesn't support the notion of a session, but

707 views • 29 slides

More recommend