Lemmatizer for Indian Languages IndiLem@FIRE-MET-2014 : An - PowerPoint PPT Presentation

IndiLem@FIRE- Setup and Indian Statistical Institute, Kolkata Abhisek Chakrabarty Lemmatizer for Indian Languages IndiLem@FIRE-MET-2014 : An Unsupervised . . . . . . Results Experimental MET-2014 : Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An December 6, 2014

IndiLem@FIRE- Experimental The proposed lemmatization Approach. Task of a lemmatizer and its need. Contents . . . . . . Results Setup and Algorithm MET-2014 : Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An Results and error analysis.

IndiLem@FIRE- Experimental The proposed lemmatization Approach. Task of a lemmatizer and its need. Contents . . . . . . Results Setup and Algorithm MET-2014 : Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An Results and error analysis.

IndiLem@FIRE- Experimental The proposed lemmatization Approach. Task of a lemmatizer and its need. Contents . . . . . . Results Setup and Algorithm MET-2014 : Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An Results and error analysis.

IndiLem@FIRE- . If we cannot map retrieved to retrieve , the meaning of Here the lemma of retrieved is retrieve . For example, ‘I retrieved that document.’ meaning of that word in that context. A lemma of a word in a context is required to retrieve the that word in that context. form of a word in context, which is known as the lemma of Lemmatization is a process that returns the base or dictionary What is lemmatization and why is it needed? . . . . . Results MET-2014 : Setup and Experimental Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An retrieved is not accessible.

IndiLem@FIRE- . If we cannot map retrieved to retrieve , the meaning of Here the lemma of retrieved is retrieve . For example, ‘I retrieved that document.’ meaning of that word in that context. A lemma of a word in a context is required to retrieve the that word in that context. form of a word in context, which is known as the lemma of Lemmatization is a process that returns the base or dictionary What is lemmatization and why is it needed? . . . . . Results MET-2014 : Setup and Experimental Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An retrieved is not accessible.

IndiLem@FIRE- . If we cannot map retrieved to retrieve , the meaning of Here the lemma of retrieved is retrieve . For example, ‘I retrieved that document.’ meaning of that word in that context. A lemma of a word in a context is required to retrieve the that word in that context. form of a word in context, which is known as the lemma of Lemmatization is a process that returns the base or dictionary What is lemmatization and why is it needed? . . . . . Results MET-2014 : Setup and Experimental Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An retrieved is not accessible.

IndiLem@FIRE- . If we cannot map retrieved to retrieve , the meaning of Here the lemma of retrieved is retrieve . For example, ‘I retrieved that document.’ meaning of that word in that context. A lemma of a word in a context is required to retrieve the that word in that context. form of a word in context, which is known as the lemma of Lemmatization is a process that returns the base or dictionary What is lemmatization and why is it needed? . . . . . Results MET-2014 : Setup and Experimental Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An retrieved is not accessible.

IndiLem@FIRE- . If we cannot map retrieved to retrieve , the meaning of Here the lemma of retrieved is retrieve . For example, ‘I retrieved that document.’ meaning of that word in that context. A lemma of a word in a context is required to retrieve the that word in that context. form of a word in context, which is known as the lemma of Lemmatization is a process that returns the base or dictionary What is lemmatization and why is it needed? . . . . . Results MET-2014 : Setup and Experimental Algorithm Lemmatization tion Lemmatiza- Basics of Chakrabarty Abhisek Languages for Indian Lemmatizer Unsupervised An retrieved is not accessible.

IndiLem@FIRE- usually store root words with their morphological and semantic . . Need of lemmatization for Indian languages Major Indian languages (Hindi, Bengali, Gujrati etc.) are morphologically very rich and suffixing in nature. Knowledge resources (dictionary, WordNet) for those languages descriptions. . Very often we face several inflected word forms in raw texts like stories, newspapers, poems etc. To obtain the meaning and morphological properties of them, we have to determine the appropriate root words which is the task of a lemmatizer. So lemmatization is necessary for building up many NLP tools . . MET-2014 : Chakrabarty An Unsupervised Lemmatizer for Indian Languages Abhisek Basics of . Lemmatiza- tion Lemmatization Algorithm Experimental Setup and Results ( WSD systems, translation systems etc.) for Indian languages.

IndiLem@FIRE- usually store root words with their morphological and semantic . . Need of lemmatization for Indian languages Major Indian languages (Hindi, Bengali, Gujrati etc.) are morphologically very rich and suffixing in nature. Knowledge resources (dictionary, WordNet) for those languages descriptions. . Very often we face several inflected word forms in raw texts like stories, newspapers, poems etc. To obtain the meaning and morphological properties of them, we have to determine the appropriate root words which is the task of a lemmatizer. So lemmatization is necessary for building up many NLP tools . . MET-2014 : Chakrabarty An Unsupervised Lemmatizer for Indian Languages Abhisek Basics of . Lemmatiza- tion Lemmatization Algorithm Experimental Setup and Results ( WSD systems, translation systems etc.) for Indian languages.

IndiLem@FIRE- usually store root words with their morphological and semantic . . Need of lemmatization for Indian languages Major Indian languages (Hindi, Bengali, Gujrati etc.) are morphologically very rich and suffixing in nature. Knowledge resources (dictionary, WordNet) for those languages descriptions. . Very often we face several inflected word forms in raw texts like stories, newspapers, poems etc. To obtain the meaning and morphological properties of them, we have to determine the appropriate root words which is the task of a lemmatizer. So lemmatization is necessary for building up many NLP tools . . MET-2014 : Chakrabarty An Unsupervised Lemmatizer for Indian Languages Abhisek Basics of . Lemmatiza- tion Lemmatization Algorithm Experimental Setup and Results ( WSD systems, translation systems etc.) for Indian languages.

IndiLem@FIRE- usually store root words with their morphological and semantic . . Need of lemmatization for Indian languages Major Indian languages (Hindi, Bengali, Gujrati etc.) are morphologically very rich and suffixing in nature. Knowledge resources (dictionary, WordNet) for those languages descriptions. . Very often we face several inflected word forms in raw texts like stories, newspapers, poems etc. To obtain the meaning and morphological properties of them, we have to determine the appropriate root words which is the task of a lemmatizer. So lemmatization is necessary for building up many NLP tools . . MET-2014 : Chakrabarty An Unsupervised Lemmatizer for Indian Languages Abhisek Basics of . Lemmatiza- tion Lemmatization Algorithm Experimental Setup and Results ( WSD systems, translation systems etc.) for Indian languages.

IndiLem@FIRE- usually store root words with their morphological and semantic . . Need of lemmatization for Indian languages Major Indian languages (Hindi, Bengali, Gujrati etc.) are morphologically very rich and suffixing in nature. Knowledge resources (dictionary, WordNet) for those languages descriptions. . Very often we face several inflected word forms in raw texts like stories, newspapers, poems etc. To obtain the meaning and morphological properties of them, we have to determine the appropriate root words which is the task of a lemmatizer. So lemmatization is necessary for building up many NLP tools . . MET-2014 : Chakrabarty An Unsupervised Lemmatizer for Indian Languages Abhisek Basics of . Lemmatiza- tion Lemmatization Algorithm Experimental Setup and Results ( WSD systems, translation systems etc.) for Indian languages.