Answer Set Programming and Human-Robot Interactions: Challenges and - PowerPoint PPT Presentation

Answer Set Programming and Human-Robot Interactions: Challenges and Opportunities Tran Cao Son Computer Science, New Mexico State University, Las Cruces, NM (Joint work with: Chitta Baral, Michael Gelfond, and Arindam Mitra) Dresden 2016 Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 1 / 29

Outline Motivation 1 Answer Set Programming 2 Addressing the Challenges 3 Extension to Agent Architecture NLP to Goal Description Planning with Goal Description What To Do When Planner Fails? Conclusions and Future Work 4 Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 2 / 29

Motivation Outline Motivation 1 Answer Set Programming 2 Addressing the Challenges 3 Extension to Agent Architecture NLP to Goal Description Planning with Goal Description What To Do When Planner Fails? Conclusions and Future Work 4 Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 3 / 29

Motivation General Intelligent Agent Architecture Given a goal Goal Create a plan 1 Deliberative agent Planner Execute the first action 2 Observe the environment, if 3 Execution Monitoring observations are as expected continue with the plan; otherwise, back to 1. Success/Fail Characteristics Communication are done via logical formulas. No interaction between agents and human during the execution of a goal. Response to a given goal is either success or fail. Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 4 / 29

Motivation General Intelligent Agent Architecture Given a goal Goal Create a plan 1 Deliberative agent Planner Execute the first action 2 Observe the environment, if 3 Execution Monitoring observations are as expected continue with the plan; otherwise, back to 1. Success/Fail Human robot interaction is important! Several practical applications: emergency rescue missions (helicopter, robots) fire fighters (robots) self-driving cars, assisted robots assistant bots (software, hardware, etc.) ... Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 4 / 29

Motivation Human-Robot Interaction Human Robot Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Add another stack to the table! Commands ✲ Human Robot Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Planner ✻ Goal Add another stack to the table! Commands ✲ Human Robot Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Planner ✻ Goal Plan Add another stack to the table! ❄ Commands ✲ Human Robot Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Planner ✻ Goal Plan Add another stack to the table! ❄ Commands ✲ Human Robot ✻ Success Plan ❄ Execution Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Planner ✻ Goal Plan Add another stack to the table! ❄ Commands ✲ ✛ Human Robot Done ✻ Success Plan ❄ Execution Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Human-Robot Interaction Planner ✻ Goal No Plan Add another blue stack of the same height! ❄ Commands ✲ ✛ Human Robot Not Done Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Motivation Challenges Planner ✻ Goal No Plan Add another blue stack of the same height! ❄ Commands ✲ ✛ Human Robot Not Done What to do if planner fails? How to communicate in NLP? (NLP commands to goals) Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 5 / 29

Answer Set Programming Outline Motivation 1 Answer Set Programming 2 Addressing the Challenges 3 Extension to Agent Architecture NLP to Goal Description Planning with Goal Description What To Do When Planner Fails? Conclusions and Future Work 4 Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 6 / 29

Answer Set Programming Logic Programming Rules and Programs Rule r : b 1 or . . . or b m ← a 1 , . . . , a n , not a n + 1 , . . . , not a n + k a i , b j : atom of a first order language L . not a : a negation-as-failure atom (naf-atom). Program A program is a set of rules. Use Herbrand semantics. Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 7 / 29

Answer Set Programming Answer Set Semantics Let π be a program and X be a set of atoms, by π X we denote the program obtained from ground ( π ) by Deleting from ground ( π ) any rule a ← a 1 , . . . , a n , not a n + 1 , not a n + k 1 for that { a n + 1 , . . . , a n + k } ∩ X � = ∅ , i.e., the body of the rule contains a naf-atom not a l and a l belongs to X ; and Removing all of the naf-atoms from the remaining rules. 2 Definition A set of atoms X is called an answer set of a program π if X is the minimal model of the program π X . Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 8 / 29

Answer Set Programming General Methodology Problem Solutions Modeling Interpretation Logic Answer Set Answer Sets Program Solver Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 9 / 29

Answer Set Programming General Methodology Traditional Problem Solutions Approach Modeling Interpretation Logic Answer Set Answer Sets Program Solver Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 9 / 29

Answer Set Programming Answer Set Planning: Block World Domain Typical ASP encoding: %Defining the time constants and objects time(0..length). block(a). block(b). block(c). %Defining fluents fluent(on(X,Y)):- block(X), block(Y),X!=Y. fluent(onTable(X)):- block(X). fluent(clear(X)):- block(X). ... %Defining actions action(stack(X,Y)):- block(X), block(Y), X!=Y. a action(putdown(X)):- block(X). ... c b %Action effects & precondition a c b :- not holds(clear(X), T), occ(pickup(X), T). holds(-onTable(X), T+1):- occ(pickup(X), T). Start Goal holds(holding(X), T+1):- occ(pickup(X), T). .... %Planning holds(on(c,b), 0). holds(onTable(b), 0). ... 1 {occ(A,T) : action(A) } 1 :- time(T). goal(T) :- holds(on(a,b), T), holds(on(b,c), T), holds(onTable(c), T). :- not goal(length). Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 10 / 29

Addressing the Challenges Outline Motivation 1 Answer Set Programming 2 Addressing the Challenges 3 Extension to Agent Architecture NLP to Goal Description Planning with Goal Description What To Do When Planner Fails? Conclusions and Future Work 4 Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 11 / 29

Addressing the Challenges Extension to Agent Architecture Idea Agents need to be able to communicate in natural language; and deal with planning failure by informing the human with what is missing so human can help. Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 12 / 29

Addressing the Challenges Extension to Agent Architecture Idea Agents need to be able to communicate in natural language; and deal with planning failure by informing the human with what is missing so human can help. Failure Response Analysis Analysis Fail NLP Commands ASP Human Planner Module Success Execution Results Results Monitoring Our Proposal: Intelligent Agent Architecture Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 12 / 29

Addressing the Challenges Extension to Agent Architecture Preliminary Results Develop a preliminary system for translating commands to NLP using kparser and NL2KR. Define the notion of a planning failure analysis: why does the planner fail to generate a plan? Implement a system for computing most preferred failure analysis. Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 13 / 29

Addressing the Challenges NLP to Goal Description NLP to Goal Description: kparser and NL2KR kparser: Semantics parser rich vocabulary for relations (e.g., event-entity, inter-event relations) relations such as prototype_of and instance_of conceptual classes of nodes based on word sense disambiguation (WordNet senses of the words represented by those nodes) semantic roles of the entities present in the text Named entity tagging and syntactic dependency parsing Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 14 / 29

Addressing the Challenges NLP to Goal Description NL2KR Sentence Meaning Add another blue stack λ x . goal_cond(x, op, add) ∧ goal_cond(x, of the same height. is, stack) ∧ goal_cond(x, color, blue) ∧ goal_cond(x, height, same) ∧ goal_cond( x, type, another). Take my blocks. λ x . add_block( has_robot(x) , has(human, x) ∧ block(x)). How about a red stack of λ x . goal_cond( x, is, stack ) ∧ goal_cond( the same height as the x, color, red ) ∧ goal_cond(x, height, same blue stack? ) ∧ goal_cond(x, origin, blue ) ∧ goal_cond( x, type, another). Add another stack of λ x . goal_cond(x, op, add) ∧ goal_cond(x, the same height as the is, stack) ∧ goal_cond(x, height, tallest) tallest stack. ∧ goal_cond(x, type, another). λ -expressions can be translated into any KR-language. Tran Cao Son (NMSU) LP & NLP & Planning Dresden 2016 15 / 29