A Language for Spoken Dialogue Management & Natural Language Processing William Yang Wang Chia-che Tsai Xin Chen Zhou Yu Department of Computer Science, Columbia University COMS W4115 Programming Languages and Translators 1
Agenda • Overview: What is Spoke? • Tutorial: How to write Spoke? • Implementation: What is inside Spoke? • Summary & Lesson Learned • Demonstration: Let’s try Spoke! 2
Where is Spoke? Automatic Speech Spoke Speech Synthesizer Recognizer Text Text Answer Question 3
Why Spoke? Motivation: “It is difficult to correctly understand the intention of speakers.” “Deep linguistic analysis is needed to resolve ambiguities.” “Traditional programming language is inefficient for analyzing NL syntax.” JAVA SPOKE Select lib, find data, configure, All in one line. train model, create structure, NL-specific data structure. run, save result ….. Advanced Pattern Matching. 4
Language Features Natural Language Syntax Tree Advanced Pattern Matching Built-in NLP APIs Simon says, Computer says, I am smart!! ( PRP I) ( VP ( VB am) ( ADJ smart)) 5
Language Features Natural Language Syntax Tree Advanced Pattern Matching Simon says, Built-in NLP APIs I am telling a truth that I am smart !! (PRP I) (VP (VB am) Computer says, Ok, you are (ADJ *)) smart. 6
Language Features Natural Language Syntax Tree Advanced Pattern Matching Built-in NLP APIs POS Tagger, Parser, Chunker, Named- entity Tagger … 7
Language Overview Hello, Jenny 1 greeting = "Hello" 2 3 func print_name(name) 4 global greeting 5 print(greeting, name, "\n") 6 end 7 8 parsed = nlpparse(readline()) 9 if parsed ~ \ I am Jenny 10 `(N(PRP I))(V(V am)(N *))` then 11 name = myobj(match[0][1][1]) 12 print_name(name) 13 fi 8
Language Overview Language Style: 1 greeting = "Hello" Script Language 2 Function Easy to write 3 func print_name(name) 4 global greeting Scoping: 5 print(greeting, name, "\n") Global Global Variable 6 end Built-in Local Variable 7 8 parsed = nlpparse(readline()) API 9 if parsed ~ \ Domain Focus: 10 `(N(PRP I))(V(V am)(N *))` then 11 name = myobj(match[0][1][1]) Built-in API 12 print_name(name) NLP operations 13 fi NLP operations 9
Language overview NLP operations: Syntax Tree Standard Patte Sta ttern Matching: char based Advanced Patte ttern Matching: synt ntax based (Sp (Spoke) …… Partial Matching Syntax Tree Representation if if pars rsed ~ `(N `(NP(NN *)(N *)(NN *)) *))(VP *)` *)` th then (Tag Word) How much did id Dow ow Jo Jone nes dro rop yeste terday? 10
Language overview NLP operations: Syntax Tree Input = ‘How much did the Dow Jones drop today?’ Pars arsed = pa parse (Input) # Par arsed = `(RO ROOT Root (SBARQ (WH WHADJP JP (WR WRB ho how) (JJ J mu much)) (SQ Q (VB VBD di did) d) SBARQ (NP P (NN do dow) (NNS jones)) (VP VP (VB VB dr drop) WHADJP SQ (NP P (NN today) y)))))) (WRB how) (JJ much) (VBD did) NP VP (NN dow) (NNS Partial Matching (Tag Word) jones) (VB (NN drop) today) if if pars rsed ~ `(N `(NP(NN *)(N *)(NN *)) *))(VP *)` *)` th then 11
Language Implementation Scanner Scanner Translator Translator Translator Parser to to to Javac IR Java Ast Java Syntax Checker 1 greeting = "Hello" ID(greeting) ASSIGN STRING(Hello) 2 EOL EOL FUNCTION ID(print_name) 3 func print_name(name) LPAREN ID(name) RPAREN EOL 4 global greeting GLOBAL ID(greeting) EOL ID(print) 5 print(greeting, name, "\n") LPAREN ID(greeting) COMMA ID(name) 6 end COMMA STRING(\n) RPAREN EOL END 7 EOL EOL ID(prsed) ASSIGN 8 parsed = nlpparse(readline()) ID(nlpparse) LPAREN ID(readline) 9 if parsed ~ \ LPAREN RPAREN RPAREN EOL IF 10 `(N(PRP I))(V(V am)(N *))` then ID(parsed) BELONG LTPAREN 11 name = myobj(match[0][1][1]) WORD(N) LTPAREN WORD(PRP) 12 print_name(name) WORD(I) RTPAREN RTPAREN 13 fi … … 12
Language Implementation Scanner Scanner Translator Translator Translator Parser Parser to to to Javac IR Java Ast Java Syntax Checker 1 greeting = "Hello" print_name: (args: name) 2 (global:greeting) (local:) 3 func print_name(name) Expr(Call(print 4 global greeting Var(greeting) 5 print(greeting, name, "\n") Var(name) 6 end String("\n"))) 7 8 parsed = nlpparse(readline()) __global__: greeting 9 if parsed ~ \ __local__: parsed name 10 `(N(PRP I))(V(V am)(N *))` then __main__: 11 name = myobj(match[0][1][1]) Expr(Assign(greeting 12 print_name(name) String("Hello"))) 13 fi … … 13
Language Implementation Scanner Scanner Translator Translator Translator Parser Parser to to to Javac IR Java Ast Java Syntax Checker Syntax Checker 1 greeting = "Hello" 2 Syntax Check 3 func print_name(name) Continue / Break not in 4 global greeting loops? 5 print(greeting, name, "\n") Return not in functions? 6 end Built-in vars assigned? 7 … and so on 8 parsed = nlpparse(readline()) 9 if parsed ~ \ Function w/o return? 10 `(N(PRP I))(V(V am)(N *))` then 11 name = myobj(match[0][1][1]) 12 print_name(name) 13 fi 14
Language Implementation Scanner Scanner IR Java Ast Java Parser Parser Javac Translator Translator Translator Syntax Checker Syntax Checker 1 greeting = "Hello" print_name: (args: name) 2 (global:greeting) (local:) 3 func print_name(name) Expr(Call(print 4 global greeting Var(greeting) 5 print(greeting, name, "\n") Var(name) 6 end String("\n"))) 7 Return(Null) 8 parsed = nlpparse(readline()) __global__: greeting 9 if parsed ~ \ __local__: parsed name 10 `(N(PRP I))(V(V am)(N *))` then __main__: 11 name = myobj(match[0][1][1]) Expr(Assign(greeting 12 print_name(name) String("Hello"))) 13 fi … … 15
Language Implementation Scanner Scanner IR Java Ast Java Parser Parser Javac Translator Translator Translator Syntax Checker Syntax Checker 1 greeting = "Hello" Ll 0 1 t0 = l1 2 ... 3 func print_name(name) Hs 1 0 3 t1 = t0 ~ t3 4 global greeting Br 1 17 0 If(17 lines) 5 print(greeting, name, "\n") Wp 0 -1 6 end Lg 1 0 arg(builtin) arg(builtin) 7 ... → l0 → l0 8 parsed = nlpparse(readline()) parsed → l1 name → l1 9 if parsed ~ \ name → l2 print_name → f0 10 `(N(PRP I))(V(V am)(N *))` then myobj (builtin ) → f140 11 name = myobj(match[0][1][1]) Match (builtin ) → g1 12 print_name(name) star (builtin ) → g2 13 fi greeting → g3 16
Language Implementation Scanner Scanner Translator Translator IR Java Ast Java Parser Parser to to Javac Translator Translator Translator IR Java Ast Syntax Checker Syntax Checker Gl 3 Global(3) JClass(greeting,SpokeProgram,public) JField(g_2,SpokeObject,private) Fn 17 f0: Func(17) JMethod(main,void,public,args:String[]) Lc 2 Local(2) JMethod(f_0,SpokeObject,public,l_0:SpokeObject) Ll 0 0 t0=l0 JDef(b,boolean) ... JDef(t_0,SpokeObject) Rt -1 Return JNew(t_0,SpokeNull) ... Lc 3 Local(3) JReturn() Cs 0 Hello t0=“Hello” JMethod(Init,void,public,l_0:SpokeObject) ... Sg 0 2 g2=t0 JNew(t_0,SpokeString,"Hello") ... JInvoke(g_2.Assign,t_0) ... Wp 1 3 t1=[t3] t_1 = JInvoke(SpokeList.Wrap,t_3) Ad 0 0 1 t0=t0+t1 t_0 = JInvoke(t_0.Operation,OpAdd,t_1) Cl 1 140 0 t1=f140(t0) t_1 = JInvoke(SpokeApi.SpokeObj,t_0) ... Sl 1 2 t2=t1 17
Language Implementation Scanner Scanner Translator Translator IR Java Ast Java Parser Parser to to Javac Translator Translator Translator IR Java Ast Syntax Checker Syntax Checker Gl 3 Global(3) JClass(greeting,SpokeProgram,public) JField(g_2,SpokeObject,private) Fn 17 f0: Func(17) JMethod(main,void,public,args:String[]) Lc 2 Local(2) JMethod(f_0,SpokeObject,public,l_0:SpokeObject) Ll 0 0 t0=l0 JDef(b,boolean) ... JDef(t_0,SpokeObject) b = JInvoke(g_2.IsCompatible,t_1) Rt -1 Return JNew(t_0,SpokeNull) ... JIf(b, Lc 3 Local(3) JReturn() JInvoke(g_2.Assign,t_1), Cs 0 Hello t0=“Hello” JMethod(Init,void,public,l_0:SpokeObject) g_2 = JInvoke(t_1.Clone,)) ... Sg 0 2 g2=t0 JNew(t_0,SpokeString,"Hello") ... JInvoke(g_2.Assign,t_0) ... Wp 1 3 t1=[t3] t_1 = JInvoke(SpokeList.Wrap,t_3) Ad 0 0 1 t0=t0+t1 t_0 = JInvoke(t_0.Operation,OpAdd,t_1) Cl 1 140 0 t1=f140(t0) t_1 = JInvoke(SpokeApi.SpokeObj,t_0) ... Sl 1 2 t2=t1 18
Recommend
More recommend
