Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network ππππππ§πππ πβππ£ β , ππ£ ππ β , πΈππ¦ππππ πΈπππ , ππ πππ£ , ππππ π·βππ , πππ§ππ πππ πβππ , πΈπππβππ ππ£ and πΌπ£π ππ£ Baidu β Natural Language Processing
Background β’ Naturally and consistently converse with human-beings on open-domain topics. β’ Data-driven β’ Retrieval-based method β’ Generation-based method β’ System ensemble Chatbots
Context-Response Matching xt π Cont ontext π(ππ‘_ππππ|π, π ) ponse π Respons
Retrieval-based Chatbot xt π Cont ontext Response π Response π A A input Response π Response π B C Search Rank Retrieval System Response π Response π D B Response π Response π C D Dialogue Corpus π(ππ‘_ππππ|π, π ) An information retrieval approach to short text conversation. Ji et al., 2014
Adversarial Dialogue Generation Dialogue π(ππ‘_ππππ|π, π ) Corpus Real Conversation Sampled <c, r> Discriminator Generator π»(π β |π) Adversarial Learning for Neural Dialogue Generation. Li et al., EMNLP-2017
Challenges Cha hallenge nges β’ Detecting truly matched segment pairs across context and response. β’ Segment pairs could be matched at different granularities. β’ Segment pairs, across context and response, could be matched because of textual relevance or semantic dependency.
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Matchi hing w ng with s h sur urface t text xt Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Matchi hing w ng with de h depe pende ndenc ncy Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Motivation Previ vious ous w wor orks ks Use GRU/LSTM to encode segments and match context with response only considering textual relevance. Mot otiva vation on β’ Self-Attent Se ntion on : Using intra-attention of utterance/response to gradually construct multi-grained semantic representations. β’ Cros oss-Attent ntion on : Using attention across context and response to match with dependency information.
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? dkpg + ` dkpg ``itβ Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance it + + ` ``dpkg dpkgβ as I donβt know the debian package manager
Conversation Context Speaker A: Hi I am looking to see what packages are installed on my system, I donβt see a path, is the list being held somewhere else? dkpg + ` dkpg ``itβ Speaker B: Try dpkg - get-selections Speaker A: What is that like? A database for packages instead of a flat file structure? Speaker B: dpkg is the debian package manager - get-selections simply shows you what packages are handed by it Response of Speaker A: No clue what do you need it for, its just reassurance it + + ` ``dpkg dpkgβ as I donβt know the debian package manager
Input nput Attentive Module Q = [ e 0 , ..., e n Q β 1 ] K = [ e 0 , ..., e n K β 1 ] V = [ e 0 , ..., e n V β 1 ] Attent ntive veModul odule(Q,K ,K,V ,V) V att = softmax ( Q Β· K T Sum um & Nor orm β’ ) V β d k β’ Fe Feed-For orward Q att = LayerNorm ( V att + Q ) β’ Sum um & Nor orm FFN ( Q att ) = max (0 , Q att W 1 + b 1 ) W 2 + b 2 Weight ghted d Sum um Attent ntion on β’ LayerNorm ( FFN ( Q att ) + Q att ) query que ke key va value ue Sum ummary β’ Capture structures across Q and K-V β’ Composite semantic representations of captured structures with input embedding Attention is All You Need. Vaswani et al., NIPS-2017
π π π π π π π Wor ord Embe beddi dding ng
Recommend
More recommend
