third person visual imitation learning via decoupled
play

Third-Person Visual Imitation Learning via Decoupled Hierarchical - PowerPoint PPT Presentation

Oct 10, 2022 •346 likes •566 views

Third-Person Visual Imitation Learning via Decoupled Hierarchical Control Pratyusha Sharma, Deepak Pathak, Abhinav Gupta Problem / Goal Can our robot manipulate a new object given a single human video alone? Why is it hard? Inferring

  1. Third-Person Visual Imitation Learning via Decoupled Hierarchical Control Pratyusha Sharma, Deepak Pathak, Abhinav Gupta

  2. Problem / Goal Can our robot manipulate a new object given a single human video alone?

  3. Why is it hard? ● Inferring useful information in the video ● Handling domain shift ● Every major part of the sequence needs to be executed correctly - Ex: For pouring, it needs to reach the cup before twisting its hand ● The manipulation is challenging. (6D, novel objects and positioning, no force feedback)

  4. Issue Scenario 1: Sequentially predict the states of the robot arm Output Input: Human demonstration + first image of object Issue: Not closed loop. No understanding of how the positions of the objects placed in front of the robot change with time!

  5. Issue Scenario 2: Sequentially predict the states of the robot arm Output Input: Human Demo + Robot visual state How do we force it to use task information from Human demonstration alone but condition its action on current observable state?

  6. We want to build a model that can infer the intent from the Human Demonstration of a task and act in the Robot’s current environment to then accomplish the task.

  7. Approach Training Goal Generator 
 (high-level) We decouple the task of Goal Inference from Local Control Controller 
 (low-level) a t

  8. Training and Test Scenarios - Data Availability Training Test (deployment) ● Human demo video ● Human demo video ● Robot demo video ● Current visible image ● Robot joint angles of the table

  9. Approach - Training Training Goal Generator: Given human demo and Goal present visual state of Generator 
 (high-level) the robot we hallucinate the next step

  10. Approach - Training Training Goal Generator: Given human demo and present visual state of Goal Generator 
 (high-level) the robot we hallucinate the next step Inverse Model : Use the hallucinated Controller 
 prediction with the (low-level) current visual state to predict the action! a t

  11. Train Time:The Goal Generator and Inverse Model are trained separately Test Time: The Goal Generator and Inverse Model are executed alternatingly

  12. Approach - Test

  13. Approach - Train Vs Test

  14. Experiments and Results We evaluate the models trained as follows: ● Goal generation model with a perfect inverse model ● Inverse model with a perfect goal generation model ● Goal generation model and inverse model in tandem

  15. Results: Goal generation model with perfect inverse model

  16. Results: Inverse model with perfect goal generator GT trajectory Predicted trajectory from GT-images

  17. Results: Final experiment runs

  18. Results: Final Experimental Runs : Placing in a box

  19. Shortcomings: 1. Robot trajectory is shaky: The robot trajectory looks shaky because of the absence of any temporal knowledge. Though trajectories predicted by inverse models with memory units(LSTM) look far less shaky but the models then over fit to the task

  20. Thank you!

Recommend

InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Chih-Hui Ho, Chun Hu,

InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Chih-Hui Ho, Chun Hu,

InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Chih-Hui Ho, Chun Hu, Po-Jung Lai 1 Outline 1. Introduction 2. Related work Generative adversarial imitation learning (GAIL) 3. Proposed method 4. Experiment

692 views • 14 slides
Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo

Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo

Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo Ciliberto, Pierluigi Amadori, Yiannis Demiris ICML 2019 Imitation Learning Teacher Student Policy learning from

275 views • 11 slides
Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement &

Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement &

Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement & Imitation) Goal: Find Optimal Policy State/Context s t Agent Imitation Learning: Optimize imitation loss Reinforcement Learning: Optimize

547 views • 53 slides
Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR

Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR

Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR Trajectory Optimization Paper Imitation Learning Supervised Learning Dagger How to solve Optimal Control Problems? Sequential Quadratic

750 views • 31 slides
CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation Learning Instructor: Animesh Garg TA: Dylan Turpin & Tingwu Wang Agenda Invitation to Imitation DAGGER: Dataset Aggregation

904 views • 41 slides
Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1 ICML-99 Slides: Dana Dahlstrom CSE 254, UCSD 2002.04.23 Overview Learning by imitation entails watching a mentor perform a task. Slide 2

414 views • 14 slides
One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas

One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas

One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba Motivation & Problem - Imitation Learning commonly applied to isolated tasks - Desire:

555 views • 21 slides
Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an

Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an

24/03/2015 Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an action form Affordance learning What is imitation? Overimitation copying unnecessary actions 2 Antonia Hamilton UCL 3

542 views • 5 slides
to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon

to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon

Reduction of Imitation Learning to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon Imitation Learning Machine Expert Learning Policy Demonstrations Algorithm 2 Imitation Learning Many

811 views • 46 slides
Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides:

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides:

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides: Dana Dahlstrom CSE 254, UCSD 2002.04.23 1 Overview In imitation, a learner observes a mentor in action. The approach proposed in this paper

440 views • 30 slides
Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation

Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation

Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation Basics and Definition Approaches & Examples Conclusion Nguyen, Thi Linh Chi Imitation Learning 2 Motivation Imitation Learning

587 views • 26 slides
A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok

A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok

A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok Jeon @ KAIST Joint work with Seokin Seo @ KAIST Kee-Eung Kim @ KAIST & PROWLER.io Imitation Learning A Markov decision process (MDP)

431 views • 14 slides
Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent

Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent state reward action S t R t A t R t+ 1 Environment S t+ 1

1.07k views • 52 slides
10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018 Recommended readings: Used Materials Much of the material and slides for this lecture were borrowed from Katerina Fragkiadaki, and Ruslan

1.25k views • 49 slides
Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Lecture 7: Imitation Learning 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2018 2 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning 3 Emma Brunskill (CS234 Reinforcement Learning. ) Winter 2018 1 /

1.27k views • 45 slides
Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2020 1 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill (CS234

797 views • 57 slides
Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2020 1 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill (CS234

526 views • 52 slides
CS885 Reinforcement Learning Module 3: July 5, 2020 Imitation Learning Torabi, F., Warnell, G.,

CS885 Reinforcement Learning Module 3: July 5, 2020 Imitation Learning Torabi, F., Warnell, G.,

CS885 Reinforcement Learning Module 3: July 5, 2020 Imitation Learning Torabi, F., Warnell, G., & Stone, P. (2018). Behavioral cloning from observation. In IJCAI (pp. 4950-4957). Ho, J., & Ermon, S. (2016). Generative adversarial

464 views • 14 slides
Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement

Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement Learning Probability Dynamics distribution over next Model T

911 views • 66 slides
Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement

Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Maximum Entropy Inverse RL, Adversarial imitation learning Katerina Fragkiadaki Reinforcement Learning Probability Dynamics distribution over next Model T

608 views • 56 slides
Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Lecture 7: Imitation Learning in Large State Spaces 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2019 2 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning in Large State Spaces 3 Emma Brunskill (CS234

924 views • 47 slides
Gen enerativ erative e Adver ersaria sarial l Im Imitation itation Le Learning arning

Gen enerativ erative e Adver ersaria sarial l Im Imitation itation Le Learning arning

Gen enerativ erative e Adver ersaria sarial l Im Imitation itation Le Learning arning Stefan efano o Ermon on Joint work with Jayesh Gupta, Jonathan Ho, Yunzhu Li, Hongyu Ren, and Jiaming Song Reinforcement Learning Goal: Learn

530 views • 40 slides
How did we get started? Learning Related Visual Problems in Baltimore City; Did I write a

How did we get started? Learning Related Visual Problems in Baltimore City; Did I write a

How did we get started? Learning Related Visual Problems in Baltimore City; Did I write a great proposal? A Long-Term Program Did I take a meeting with an influential person? Did I respond to a request for proposals Paul

544 views • 26 slides
Imitation Learning from Imperfect Demonstration Yueh-Hua Wu 1,2 , Nontawat Charoenphakdee 3,2 , Han

Imitation Learning from Imperfect Demonstration Yueh-Hua Wu 1,2 , Nontawat Charoenphakdee 3,2 , Han

Imitation Learning from Imperfect Demonstration Yueh-Hua Wu 1,2 , Nontawat Charoenphakdee 3,2 , Han Bao 3,2 , Voot Tangkaratt 2 , Masashi Sugiyama 2,3 1 National Taiwan University 2 RIKEN Center for Advanced Intelligence Project 3 The University of

309 views • 12 slides

More recommend