Machine Translation CIS 526 Instructor: Chris Callison-Burch TAs: - PowerPoint PPT Presentation

Machine Translation CIS 526 Instructor: Chris Callison-Burch TAs: Mitchell Stern, Justin Chiu

Course web site mt-class.org/penn Course materials developed with Adam Lopez Matt Post Chris Dyer Edinburgh JHU CMU

Textbook

ندلب نب ہماسا ںیم ہک ےہ انہک اک ینیچ کڈ ردص یکیرما بئان ۔ںوہ اتہاچ انھکید ہدرم ای ہدنز وک American Vice President Dick Cheney has said that he wants to see Osama bin Laden dead or alive.

The Tower of Babel Pieter Brueghel the Elder (1563)

ENIAC (1946)

When I look at an article in Russian, I say: “This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.” Warren Weaver (1949)

Star Trek Hitchhiker’s Guide to the Galaxy

Statistical Machine Translation Live 4/28/2006 Franz Och Because we want to provide everyone with access to all the world's information, including information written in every language, one of the exciting projects at Google Research is machine translation... Now you can see the results for yourself. We recently launched an online version of our system for Arabic-English and English-Arabic. Try it out!

Statistical Machine Translation Develop a statistical model of translation that can be learned from data and used to predict the correct English translation of new Chinese sentences.

Statistical Machine Translation information theory formal language linguistics machine learning algorithms

In-class exercise

Synchronous ¡Context ¡Free ¡Grammar Urdu English S → NP ① VP ② NP ① VP ② VP → PP ① VP ② VP ② PP ① VP → V ① AUX ② AUX ② V ① PP → NP ① P ② P ② NP ① NP → Hamid Ansari hamd ansary NP → Vice President na}b sdr V → nominated namzd P → for kylye AUX → was taa 16

NP ❶ NP ❷ P ❸ V ❹ AUX ❺ hamd ansary na}b sdr kylye taa namzd P ❸ NP ❶ NP ❷ V ❹ AUX ❺ for Hamid Ansari nominated was Vice President

PP ❻ NP ❶ NP ❷ P ❸ V ❹ AUX ❺ hamd ansary na}b sdr kylye taa namzd PP ❻ P ❸ NP ❶ NP ❷ V ❹ AUX ❺ for Hamid Ansari nominated was Vice President

VP ❼ PP ❻ NP ❶ NP ❷ P ❸ V ❹ AUX ❺ hamd ansary na}b sdr kylye taa namzd VP ❼ PP ❻ NP ❶ P ❸ NP ❷ V ❹ AUX ❺ Hamid Ansari nominated was for Vice President

VP ❽ VP ❼ PP ❻ NP ❶ NP ❷ P ❸ V ❹ AUX ❺ hamd ansary na}b sdr kylye taa namzd VP ❽ VP ❼ PP ❻ NP ❶ P ❸ NP ❷ AUX ❺ V ❹ Hamid Ansari was for Vice President nominated

S ❾ VP ❽ VP ❼ PP ❻ NP ❶ NP ❷ P ❸ V ❹ AUX ❺ hamd ansary na}b sdr kylye taa namzd S ❾ VP ❽ VP ❼ PP ❻ NP ❶ P ❸ NP ❷ AUX ❺ V ❹ Hamid Ansari for Vice President was nominated

Transla7on ¡improvements First nuclear test conducted in 1990 The First Nuclear Test Was in 1990. ﻼﮩﭘ ’ 'first nuclear experiment in 1990 Thomas Reed, who has worked as a was' Thomas red of the United States, the weapons designer at Livermore Thomas red Unilever National ﻣﺎﮭﻭﺗ National Laboratory in designer are National Laboratory in the United Laboratory of the United States ﭨﮣرﺮﺎﺑﯾﻴﻟ already working on the book of Los States, has written a book in in نﻦﭘﯾﻴوﻮ designer, are already working Alamos National Laboratory, former collaboration with Danny Stillman, ﭼرﺮﮐ on the book of Los سﺲوﻮﻣﻠﯾﻴاﺎ National director of the technical intelligence, former director of the technical ﭨﮣرﺮﺎﺑﯾﻴﻟ Laboratory ﯽﻧﯾﻴڈﮉ , former director of the with the cooperation of Diana intelligence division at Los Alamos ﯾﻴرﺮﺋاﺎڈﮉ technical سﺲﻧﺟﻠﯾﻴﭨﮣﻧاﺎ written with the steelman wrote. National Laboratory. cooperation of نﻦﯾﻴﻣﻠﭨﮣﺳ . ۔ﮯﮨ This book under the title of the spread In their book, 'The Nuclear Express: A ﺗﮐ ہﮨﯾﻴ This book 'nuclear express: political of nuclear expressway: the political Political History of the Bomb and its ﯾﻴرﺮﺎﺗ history and the expansion of bomb' history of the bomb and this has been Proliferation,' Reed and Stillman claim has been written, and the two writers ﯽﮭﻭﮑﻟ written and the two writers have that every country that has ever have also claimed that the country has دﺪ ﮯﻧ claimed that the country also has produced a nuclear bomb has been made nuclear bomb is he or any other made nuclear bomb or any other ﺎﻧﺑ مﻢﺑ able to do so because it stole the country's nuclear secrets to country, Korea nuclear secrets, or any nuclear secrets of another country or ﮯﮐ ﮯٔٵﯾﻴاﺎرﺮﭼ or that of any other nuclear of the other nuclear power enjoyed the cooperation of another ﯽﺳﮐ power cooperation is achieved. cooperation. 22 nuclear power. ۔ﮯﮨ ﺎﮨرﺮ

Who ¡did ¡what ¡to ¡whom? Baseline He said that China, North Korea, Iran, Syria, Pakistan, through Egypt, Libya and Yemen is to provide nuclear technology. Thomas was red when this question why China has provided the nuclear technology to Pakistan, In response, He said as China and India was joint enemy of Pakistan. Syntactic final system He said that China would provide nuclear technology to North Korea, Iran, Syria, Pakistan, Egypt, Libya and Yemen. Thomas red when was this question why China has provided to Pakistan nuclear technology, he said in response to China, Pakistan and India as a common enemy. 23

Example ¡Research ¡ques7ons • How ¡can ¡we ¡learn ¡transla7on ¡rules ¡automa7cally ¡ from ¡data? ¡ • When ¡one ¡word ¡has ¡several ¡different ¡possible ¡ transla7ons, ¡which ¡one ¡should ¡we ¡choose? ¡ • When ¡languages ¡have ¡different ¡word ¡orders, ¡ how ¡do ¡we ¡properly ¡re-­‑reorder ¡the ¡words ¡from ¡ one ¡language ¡into ¡the ¡other ¡language? ¡ • What ¡is ¡the ¡most ¡effec7ve ¡way ¡of ¡searching ¡over ¡ all ¡permuta7ons ¡and ¡combina7ons ¡of ¡words? 24

Topics ¡in ¡CIS ¡526 • Probability ¡and ¡Language ¡Models ¡ • Word ¡Alignment ¡and ¡Transla7on ¡Models ¡ • Searching ¡for ¡the ¡most ¡probable ¡transla7on ¡ • Phrase-­‑based ¡models ¡and ¡discrimina7ve ¡training ¡ • Evalua7ng ¡transla7on ¡quality ¡ • Syntax-­‑based ¡models ¡of ¡transla7on ¡ • Collec7ng ¡training ¡data ¡through ¡crowdsourcing ¡ and ¡web ¡crawling 25

Who ¡should ¡take ¡this ¡class? • Anyone ¡who ¡is ¡interested ¡in ¡machine ¡transla7on, ¡ natural ¡language ¡processing, ¡or ¡AI ¡ • I ¡don’t ¡presume ¡any ¡background ¡in ¡linguis7cs ¡or ¡ sta7s7cs ¡ • The ¡only ¡prerequisite ¡is ¡good ¡programming ¡skills ¡ • All ¡the ¡code ¡for ¡the ¡class ¡is ¡wriTen ¡in ¡Python 26

Assignments • Programming ¡assignments ¡are ¡designed ¡to ¡teach ¡ you ¡the ¡fundamental ¡algorithms ¡in ¡SMT ¡and ¡ illustrate ¡the ¡research ¡challenges 27

Word ¡aligner Garcia and associates . the clients and the associates are enemies . Garcia y asociados . los clientes y los asociados son enemigos . Carlos Garcia has three associates . the company has three groups . Carlos Garcia tiene tres asociados . la empresa tiene tres grupos . his associates are not strong . its groups are in Europe . sus asociados no son fuertes . sus grupos estan en Europa . Garcia has a company also . the modern groups sell strong pharmaceuticals . Garcia tambien tiene una empresa . los grupos modernos venden medicinas fuertes . its clients are angry . the groups do not sell zanzanine . sus clientes estan enfadados . los grupos no venden zanzanina . the associates are also angry . the small groups are not modern . los asociados tambien estan enfadados . los grupos pequenos no son modernos .

Word ¡aligner Garcia and associates . the clients and the associates are enemies . Garcia y asociados . los clientes y los asociados son enemigos . Carlos Garcia has three associates . the company has three groups . Carlos Garcia tiene tres asociados . la empresa tiene tres grupos . his associates are not strong . its groups are in Europe . sus asociados no son fuertes . sus grupos estan en Europa . Garcia has a company also . the modern groups sell strong pharmaceuticals . Garcia tambien tiene una empresa . los grupos modernos venden medicinas fuertes . its clients are angry . the groups do not sell zanzanine . sus clientes estan enfadados . los grupos no venden zanzanina . the associates are also angry . the small groups are not modern . los asociados tambien estan enfadados . los grupos pequenos no son modernos .

Phrase ¡Extractor 澳 是 与 北 韩 有 邦 的 少 国 之 洲 交 数 家 一 Australia 北 韩 
 is North Korea one of the 邦交 few countries diplomatic relations that have diplomatic 与 北 韩 有 邦交 relations with have diplomatic relations North with North Korea Korea 30