Interactive Learning of Grounded Verb Semantics towards Human-Robot - PowerPoint PPT Presentation

Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication Lanbo She and Joyce Y. Chai Department of Computer Science and Engineering Michigan State University Presenter: Yuyang Rao April 2017 Free PowerPoint Templates Free PowerPoint Templates

“ “ Human-Robot Interaction (HRI) is a field of study dedicated to understanding, designing, and evaluating robotic systems for use by or with humans. Interaction, by definition, requires communication between robots and humans. --HRI Free PowerPoint Templates Free PowerPoint Templates

Overview Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication Goal Verb Semantics Interactive Learning • Command eg: • Questions • Better Boil the water Human robot • Answers interaction Free PowerPoint Templates Free PowerPoint Templates

Introduction Challenge robots do not have sufficient linguistic or world knowledge as humans do allows robots to proactively Interactive learning engage in interaction with human partners Reward system + update Reinforcement learning knowledge base Free PowerPoint Templates Free PowerPoint Templates

Previous Work Learning Approach Disadvantage 1 Disadvantage 2 Under the assumption Learning Reply on on Each demonstration is of perfect perception multiple instances of simply a sequence of of the environment. human primitive actions However, does not demonstrations of associated with a verb. hold in real-world corresponding No other type of situated interaction. actions. interaction Free PowerPoint Templates Free PowerPoint Templates

State-based Representation How Robot understand the command: “boil the water” Free PowerPoint Templates Free PowerPoint Templates

Hypothesis Space Command Boil the water (verb phrase) Execution select a most relevant hypothesis and use the corresponding goal state to plan for actions to execute. Learning If fails, ask the human for a demonstration. Update Up Based on the demonstrated actions, the robot will learn a new representation Free PowerPoint Templates Free PowerPoint Templates

Noisy Environment Environment representation The environment representation is often partial, error prone, and full of uncertainties. No Yes Previous works Interactive assume: learning perfect, approach aims deterministic to address these representation uncertainties Free PowerPoint Templates Free PowerPoint Templates

Framework of Interactive Learning Free PowerPoint Templates Free PowerPoint Templates

The Big Question Ask Question ? When to What Free PowerPoint Templates Free PowerPoint Templates

What Questions to Ask? Free PowerPoint Templates Free PowerPoint Templates

When to? Input 1 e, l, Feature function φ; Old policy θ ; Hypotheses H; Initialize 2 state s initialized with e, l ; first action a ∼ P (a|s; θ) while s is not terminal do Take action a, receive reward r; 3 Algorithm update State s Choose aʹ ∼ P(aʹ|sʹ;θ) End 4 if s terminates with positive feedback then Update H; end 5 Output : Updated H and θ. Free PowerPoint Templates Free PowerPoint Templates

Execution Example Execution Example H1: Boil the water. H1: Boil the water. R1: I see a silver object on the table, is that a R1: ( select a hypothesis and pot? H2: Yes. execute the action accordingly ) H2: R2: Should I leave the pot on the stove after ( provide true/false feedback on boil? H3: Yes. whether the execution is correct ) R3: Ok. ( select a hypothesis to execute ) H4: Thank you. Comparison Learning Example Learning Example H3: Boil the water. H5: Boil the water. R3: I don’t know how to boil, could you R5: I don’t know how to boil, could you show show me an example? H4: ( human me an example? H6: Ok. ( demonstrate the shows a sequence of primitive actions ) task, at the end put the pot on a heat pad ) R4: ( indicate the acceptance and induce R6: So after boiling something, should I put it a hypothesis to represent verb on a heat pad? H7: No, you don’t have to. semantics ) R7: I see. Free PowerPoint Templates Free PowerPoint Templates

Expe peri riment Setup up Dataset Noisy Environment Approaches Evaluation Metrics Representation (1) Kitchen & living (1)PerfectEnv (1) IED: action (1) She 16 room; sequence (2)NormStd3 (2) RandomPolicy (2) 979 instances (3)NormStd5 (2) SJI: state changes (3) ManualPolicy (4)UniEnv Free PowerPoint Templates Free PowerPoint Templates

Result__________ 1. The interactive learning with RL policy outperforms the previous approach She16 . 2. The RL policy slightly outperforms interactive learning using manually defined policy. 3. However, the manualPolicy results in much longer interaction (i.e., more Figure 5: Performance (SJI) comparison on questions) than the RL different interaction policies to the testing data. policy. Free PowerPoint Templates Free PowerPoint Templates

Result 1. When the environment becomes noisy, the performance of She16 that only relies on demonstrations decreases significantly. 2. IL improves the performance under the perfect environment Table 1: Performance comparison between She16 and condition our interactive learning based on environment 3. Effect in noisy environment representations with different levels of noise is more remarkable. Free PowerPoint Templates Free PowerPoint Templates

Co Concl nclus usion Future Work Now To learn new Deep neural Asking intelligent Robots live in a predicates by network to questions to noisy interaction with alleviate feature interact with environment, full humans engineering human can handle of uncertainties. the uncertainties Free PowerPoint Templates Free PowerPoint Templates

Free PowerPoint Templates Free PowerPoint Templates