Lecture 1: Introduction Kai-Wei Chang CS @ UCLA kw@kwchang.net - PowerPoint PPT Presentation

Lecture 1: Introduction Kai-Wei Chang CS @ UCLA kw@kwchang.net Couse webpage: https://uclanlp.github.io/CS269-17/ ML in NLP 1

Announcements v Waiting list: Start attending the first few lectures as if you are registered. Given that some students will drop the class, some space will free up. v We will use Piazza as an online discussion platform. Please sign up here: piazza.com/ucla/fall2017/cs269 ML in NLP 2

Staff v Instructor: Kai-Wei Chang v Email: ml17@kwchang.net v Office: BH 3732J v Office hour: 4:00 – 5:00, Tue (after class). v TA: Md Rizwan Parvez v Email: wua4nw@virginia.edu v Office: BH 3809 v Office hour: 12:00 – 2:00, Wed ML in NLP 3

This lecture v Course Overview v What is NLP? Why it is important? v What types of ML methods used in NLP? v What will you learn from this course? v Course Information v What are the challenges? v Key NLP components v Key ML ideas in NLP ML in NLP 4

What is NLP v Wiki: Natural language processing ( NLP ) is a field of computer science, artificial intelligence, and computational linguistics concerned with the interactions between computers and human ( natural ) languages. ML in NLP 5

Go beyond the keyword matching v Identify the structure and meaning of words, sentences, texts and conversations v Deep understanding of broad language v NLP is all around us ML in NLP 6

Machine translation Facebook translation, image credit: Meedan.org ML in NLP 7

Statistical machine translation Image credit: Julia Hockenmaier, Intro to NLP ML in NLP 8

Dialog Systems ML in NLP 9

Sentiment/Opinion Analysis ML in NLP 10

Text Classification www.wired.com v Other applications? ML in NLP 11

Question answering 'Watson' computer wins at 'Jeopardy' credit: ifunny.com ML in NLP 12

Question answering v Go beyond search ML in NLP 13

Natural language instruction https://youtu.be/KkOCeAtKHIc?t=1m28s ML in NLP 14

Digital personal assistant More on natural language instruction credit: techspot.com v Semantic parsing – understand tasks v Entity linking – “my wife” = “Kellie” in the phone book ML in NLP 15

Information Extraction v Unstructured text to database entries Yoav Artzi: Natural language processing ML in NLP 16

Language Comprehension Christopher Robin is alive and well. He is the same person that you read about in the book, Winnie the Pooh. As a boy , Chris lived in a pretty home called Cotchfield Farm . When Chris was three years old, his father wrote a poem about him . The poem was printed in a magazine for others to read. Mr. Robin then wrote a book v Q: who wrote Winnie the Pooh? v Q: where is Chris lived? ML in NLP 17

What will you learn from this course v The NLP Pipeline v Key components for understanding text v NLP systems/applications v Current techniques & limitation v Build realistic NLP tools ML in NLP 18

What’s not covered by this course v Speech recognition – no signal processing v Natural language generation v Details of ML algorithms / theory v Text mining / information retrieval ML in NLP 19

This lecture v Course Overview v What is NLP? Why it is important? v What will you learn from this course? v Course Information v What are the challenges? v Key NLP components ML in NLP 20

Overview v New course, first time being offered v Comments are welcomed v target at first- or second- year PhD students v Lecture + Seminar v No course prerequisites, but I assume v programming experience (for the final project) v basic ML/AI background v basics of probability calculus, and linear algebra (HW0) ML in NLP 21

Grading v Attendance & participations (10%) v Participate in discussion v Paper summarization report (20%) v Paper presentation (30%) v Final project (40%) v Proposal (5%) v Final Paper (25%) v Presentation (10%) ML in NLP 22

Paper summarization v 1 page maximum v Pick one paper from recent ACL/NAACL/EMNLP/EACL v Summarize the paper (use you own words) v Write a blog post using markdown or jupyter notebook: v https://einstein.ai/research/learned-in- translation-contextualized-word-vectors v https://github.com/uclanlp/reducingbias/blob/ma ster/src/fairCRF_gender_ratio.ipynb ML in NLP 23

ML in NLP 24

ML in NLP 25

Paper presentation v Each group has 2~3 students v Read and understand 2~3 related papers v Cannot be the same as your paper summary v Can be related to your final project v Register your choice next week v 30 min presentation/ Q&A v Grading Rubric: 40% technical understanding, 40% presentation, 20% interaction ML in NLP 26

Final Project v Work in groups (3 students) v Project proposal v 1 page maximum (template) v Project report v Similar to the paper summary v Due before the final presentation v Project presentation v in-class presentation (tentative) ML in NLP 27

Late Policy v Submission site will be closed 1hr after the deadline. v No late submission v unless under emergency situation ML in NLP 28

Cheating/Plagiarism v No . Ask if you have concerns v Rules of thumb: v Cite your references v Clearly state what are your contributions ML in NLP 29

Lectures and office hours v Participation is highly appreciated! v Ask questions if you are still confusing v Feedbacks are welcomed v Lead the discussion in this class v Enroll Piazza ML in NLP 30

Topics of this class v Fundamental NLP problems v Machine learning & statistical approaches for NLP v NLP applications v Recent trends in NLP ML in NLP 31

What to Read? v Natural Language Processing ACL, NAACL, EACL, EMNLP, CoNLL, Coling, TACL aclweb.org/anthology v Machine learning ICML, NIPS, ECML, AISTATS, ICLR, JMLR , MLJ v Artificial Intelligence AAAI, IJCAI, UAI, JAIR ML in NLP 32

Questions? ML in NLP 33

This lecture v Course Overview v What is NLP? Why it is important? v What will you learn from this course? v Course Information v What are the challenges? v Key NLP components v Key ML ideas in NLP ML in NLP 34

Challenges – ambiguity v Word sense ambiguity ML in NLP 35

Challenges – ambiguity v Word sense / meaning ambiguity Credit: http://stuffsirisaid.com ML in NLP 36

Challenges – ambiguity v PP attachment ambiguity Credit: Mark Liberman, http://languagelog.ldc.upenn.edu/nll/?p=17711 ML in NLP 37

Challenges -- ambiguity v Ambiguous headlines: v Include your children when baking cookies v Local High School Dropouts Cut in Half v Hospitals are Sued by 7 Foot Doctors v Iraqi Head Seeks Arms v Safety Experts Say School Bus Passengers Should Be Belted v Teacher Strikes Idle Kids ML in NLP 38

Challenges – ambiguity v Pronoun reference ambiguity Credit: http://www.printwand.com/blog/8-catastrophic-examples-of-word-choice-mistakes ML in NLP 39

Challenges – language is not static v Language grows and changes v e.g., cyber lingo LOL Laugh out loud G2G Got to go BFN Bye for now B4N Bye for now Idk I don’t know FWIW For what it’s worth LUWAMH Love you with all my heart ML in NLP 40

Challenges--language is compositional Carefully Slide ML in NLP 41

Challenges--language is compositional 小心 : 地滑 : Carefully Slide Careful Landslip Take Wet Floor Care Smooth Caution ML in NLP 42

Challenges – scale v Examples: v Bible (King James version): ~700K v Penn Tree bank ~1M from Wall street journal v Newswire collection: 500M+ v Wikipedia: 2.9 billion word (English) v Web: several billions of words ML in NLP 43

This lecture v Course Overview v What is NLP? Why it is important? v What will you learn from this course? v Course Information v What are the challenges? v Key NLP components v Key ML ideas in NLP ML in NLP 44

Part of speech tagging ML in NLP 45

Syntactic (Constituency) parsing ML in NLP 46

Syntactic structure => meaning Image credit: Julia Hockenmaier, Intro to NLP ML in NLP 47

Dependency Parsing ML in NLP 48

Semantic analysis v Word sense disambiguation v Semantic role labeling Credit: Ivan Titov ML in NLP 49

Q: [Chris] = [Mr. Robin] ? Christopher Robin is alive and well. He is the same person that you read about in the book, Winnie the Pooh. As a boy , Chris lived in a pretty home called Cotchfield Farm . When Chris was three years old, his father wrote a poem about him . The poem was printed in a magazine for others to read. Mr. Robin then wrote a book Slide modified from Dan Roth ML in NLP 50

Co-reference Resolution Christopher Robin is alive and well. He is the same person that you read about in the book, Winnie the Pooh . As a boy , Chris lived in a pretty home called Cotchfield Farm. When Chris was three years old, his father wrote a poem about him . The poem was printed in a magazine for others to read. Mr. Robin then wrote a book ML in NLP 51

This lecture v Course Overview v What is NLP? Why it is important? v What will you learn from this course? v Course Information v What are the challenges? v Key NLP components v Key ML ideas in NLP ML in NLP 52

Machine learning 101 CS6501- Advanced Machine Learning 53

CS6501- Advanced Machine Learning 54

Perceptron, decision tree, support vector machine K-NN, Naïve Bayes, logistic regression…. CS6501- Advanced Machine Learning 55