Simple, Lexicalized Choice of Translation Timing for Simultaneous - PowerPoint PPT Presentation

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Tomoki Fujita, Graham Neubig, Sakriani Sakti, Tomoki Toda, Satoshi Nakamura Graduate School of Information Science, Nara Institute of Science and Technology 1

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Background 2

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Speech Translation Systems ● Translate speech from source language to target ASR こんにちは、駅はどこですか？ MT Hello, where is the station? TTS 3

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Problem: Delay ● Wait for the whole utterance to end before translating Delay ASR こんにちは、駅はどこですか？ MT Hello, where is the station? TTS 4

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Solution: Divide into Smaller Chunks ● Choose appropriate timing to start translation Delay: Reduced ASR こんにちは、 駅は どこですか？ MT MT MT Hello, the station where is it? TTS TTS TTS 5

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Previous Work: Incremental Dependency Parsing/Manual Rules [Ryu+ 04] ● Utilize knowledge of English/Japanese to derive rules subj prep prep I went to the park with your brother Translate after the first prepositional phrase completes! MT MT あなたの弟と 私は公園に行きました ● - Requires a bilingual linguist to design rules ● - Requires an accurate incremental dependency parser 6

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Previous Work: Division on Pauses [Fugen+ 08, Bangalore+ 12] ● Simply divide on short pauses in the utterance ASR hello where is the station ● - Cannot capture relationship between languages ● - Result will greatly change with speech speed, disfluencies 7

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Proposed Method ● Utilize the TM directly to choose translation timing ● + Can be constructed automatically ● + Uses information about the language pair ● + Very simple to implement ● Specifically: ● Choose translation timing at the end of each phrase in the phrase table ● Utilize reordering probabilities to adjust granularity ● Adapt the language model to the translation task 8

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Preliminaries 9

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Phrase Based Machine Translation ● Divide the sentence into small phrases and translate Today I will give a lecture on machine translation . Today I will give a lecture on machine translation . 今日は、 を行います の講義 機械翻訳 。 Today machine translation a lecture on I will give . 今日は、 機械翻訳 の講義 を行います 。 今日は、機械翻訳の講義を行います。 ● Score translations with translation model (TM), 10 reordering model (RM), and language model (LM)

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Translation Model Creation ● Perform automatic alignment of bitext ● From aligned text, extract phrases for translation ホ テ 受 ホテル の → hotel ルの付 ホテル の → the hotel the 受付 → front desk hotel ホテルの受付 → hotel front desk front ホテルの受付 → the hotel front desk desk

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Lexicalized Reordering Model ● Probabilistically models reorderings for increased accuracy of translation ● Given current phrase and next phrase: Monotone: Swap: 背 の 高い 男 太郎 を 訪問 した the tall man visited Taro Discontinuous Right: Discontinuous Left: 私 は 太郎 を 訪問した 背 の 高い 男 を 訪問 した I visited Taro visited the tall man ● “monotone” + “discontinuous right” = “right probability”

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Proposed Method 13

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” phrase exists ↓ wait

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” phrase exists ↓ wait

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” “hello where” phrase exists phrase missing ↓ ↓ wait translate “hello”

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” “hello where” phrase exists phrase missing ↓ ↓ wait translate “hello”

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” “hello where” “where is” phrase exists phrase missing phrase exists ↓ ↓ ↓ wait translate wait “hello”

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” “hello where” “where is” phrase exists phrase missing phrase exists ↓ ↓ ↓ wait translate wait “hello”

Simple, Lexicalized Choice of Translation Timing for Simultaneous Speech Translation Method One: Choosing Translation Timing with Phrases ● Input words one at a time from ASR ● While words exist in phrase table, don't translate yet Phrase Table hello→ こんにちは where is→ どこですか the station→ 駅 where→ どこ the→ その Input String hello where is the station “hello” “hello where” “where is” “where is the” phrase exists phrase missing phrase exists phrase missing ↓ ↓ ↓ ↓ wait translate wait translate “hello” “where is”