infogail interpretable imitation learning from visual
play

InfoGAIL: Interpretable Imitation Learning from Visual - PowerPoint PPT Presentation

May 03, 2023 •532 likes •692 views

InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Chih-Hui Ho, Chun Hu, Po-Jung Lai 1 Outline 1. Introduction 2. Related work Generative adversarial imitation learning (GAIL) 3. Proposed method 4. Experiment

  1. InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations Chih-Hui Ho, Chun Hu, Po-Jung Lai 1

  2. Outline 1. Introduction 2. Related work ○ Generative adversarial imitation learning (GAIL) 3. Proposed method 4. Experiment results 5. Conclusion 2

  3. Introduction ● A reward function is important in RL task ● Hard to design reward function in some scenario (e.g. autonomous driving) ● Imitation learning allows agents to learn how to perform task like an expert ○ Generative Adversarial Imitation Learning (GAIL, [12]) ○ Generative adversarial nets (GANs, [13]) ● Expert demonstrations varies significantly ○ Multiple experts might have multiple policies ○ Need external latent factors to better represent the observed behavior ● Goal: To develop an imitation learning framework that is able to automatically discover and disentangle the latent factors of variation underlying expert demonstrations 3

  4. GAN for imitation learning (GAIL) https://www.youtube.com/watch?v=rOho-2oJFeA 4

  5. GAN for imitation learning (GAIL) 5

  6. Proposed method ● Introduce a latent factor c to represent the variation under expert demonstrations ● In GAIL, action is chosen as ● Proposed method chooses action as ● Maximize the mutual information between latent code c and {state, action}. ● is a function of GAIL InfoGAIL 6

  7. Proposed method ● Discriminator maximizes ● Mutual information minimizes ● Policy updates with TRPO[2] 7

  8. Proposed method ● Reward augmentation ○ Helps when expert perform sub-optimally ○ Hybrid between RL and imitation learning ● Replace vanilla GAN with WGAN[26] ○ More stable and easier to train ○ 8

  9. Experiment Result - Learning to Distinguish Trajectories ● The driving experiment are conducted on Open Source Race Car Simulator ● Each color denotes one specific latent code ○ Different experts have different trajectories 9

  10. Experiment Result - Interpretable Imitation Learning ● Blue and red indicate policies under different latent codes ● They correspond to “turning from inner lane” and “turning from outer lane” respectively 10

  11. Experiment Result - Interpretable Imitation Learning ● Different latent codes correspond to passing from right or left InfoGAIL GAIL 11

  12. Experiment 12

  13. Conclusion ● Automatically distinguish certain driving behaviors by introducing the latent factors ● Discovering the latent factors without direct supervision ● Perform imitation learning by using only visual inputs ● Learning a policy that can imitate and even outperform the human experts 13

  14. Demo Video 14

Recommend

Third-Person Visual Imitation Learning via Decoupled Hierarchical Control Pratyusha Sharma,

Third-Person Visual Imitation Learning via Decoupled Hierarchical Control Pratyusha Sharma,

Third-Person Visual Imitation Learning via Decoupled Hierarchical Control Pratyusha Sharma, Deepak Pathak, Abhinav Gupta Problem / Goal Can our robot manipulate a new object given a single human video alone? Why is it hard? Inferring

566 views • 20 slides
Deep Visual Models with Interpretable Features and Modularized Structures Quanshi Zhang John

Deep Visual Models with Interpretable Features and Modularized Structures Quanshi Zhang John

Deep Visual Models with Interpretable Features and Modularized Structures Quanshi Zhang John Hopcroft Center Shanghai Jiao Tong University Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu, "Interpretable Convolutional Neural Networks"

694 views • 42 slides
Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo

Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo

Random Expert Distillation For Imitation Learning Ruohan Wang, Carlo Ciliberto, Pierluigi Amadori, Yiannis Demiris ICML 2019 Imitation Learning Teacher Student Policy learning from

275 views • 11 slides
Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement &

Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement &

Learning to Optimize as Policy Learning Yisong Yue Policy Learning (Reinforcement & Imitation) Goal: Find Optimal Policy State/Context s t Agent Imitation Learning: Optimize imitation loss Reinforcement Learning: Optimize

547 views • 53 slides
Meta-Explanations, Interpretable Clustering & Other Recent Developments Fraunhofer HHI,

Meta-Explanations, Interpretable Clustering & Other Recent Developments Fraunhofer HHI,

Fraunhofer Image Processing Heinrich Hertz Institute Meta-Explanations, Interpretable Clustering & Other Recent Developments Fraunhofer HHI, Machine Learning Group Wojciech Samek ICCV 2019 Visual XAI Workshop Seoul, Korea, 2th

1.29k views • 60 slides
IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules

IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules

IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules Bishwamittra Ghosh Joint work with Kuldeep S. Meel 1 Applications of Machine Learning 2 Example Dataset 3 Representation of an interpretable

1.24k views • 55 slides
Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR

Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR

Trajectory Optimization, Imitation Learning Lecture 14 What will you take home today? Recap LQR Trajectory Optimization Paper Imitation Learning Supervised Learning Dagger How to solve Optimal Control Problems? Sequential Quadratic

750 views • 31 slides
CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation

CSC2621 Topics in Robotics Reinforcement Learning in Robotics Week 2: Supervised & Imitation Learning Instructor: Animesh Garg TA: Dylan Turpin & Tingwu Wang Agenda Invitation to Imitation DAGGER: Dataset Aggregation

904 views • 41 slides
Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slide 1 ICML-99 Slides: Dana Dahlstrom CSE 254, UCSD 2002.04.23 Overview Learning by imitation entails watching a mentor perform a task. Slide 2

414 views • 14 slides
One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas

One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas

One-Shot Imitation Learning Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba Motivation & Problem - Imitation Learning commonly applied to isolated tasks - Desire:

555 views • 21 slides
Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an

Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an

24/03/2015 Formal terms 1 Object-learning Social imitation Emulation Mimicry copying only an action form Affordance learning What is imitation? Overimitation copying unnecessary actions 2 Antonia Hamilton UCL 3

543 views • 5 slides
to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon

to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon

Reduction of Imitation Learning to No-Regret Online Learning Stephane Ross Joint work with Drew Bagnell & Geoff Gordon Imitation Learning Machine Expert Learning Policy Demonstrations Algorithm 2 Imitation Learning Many

811 views • 46 slides
Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides:

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides:

Implicit Imitation in Multiagent Reinforcement Learning Bob Price and Craig Boutilier Slides: Dana Dahlstrom CSE 254, UCSD 2002.04.23 1 Overview In imitation, a learner observes a mentor in action. The approach proposed in this paper

440 views • 30 slides
Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation

Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation

Imitation Learning Initial Concept and Approaches Nguyen, Thi Linh Chi Outline Motivation Basics and Definition Approaches & Examples Conclusion Nguyen, Thi Linh Chi Imitation Learning 2 Motivation Imitation Learning

587 views • 26 slides
Toward In Interpretable De Deep Re Reinforcement Lea Learning g wi with Li Linea ear Model

Toward In Interpretable De Deep Re Reinforcement Lea Learning g wi with Li Linea ear Model

Toward In Interpretable De Deep Re Reinforcement Lea Learning g wi with Li Linea ear Model el U-Tr Trees Guiliang Liu, Oliver Schulte, Wang Zhu, Qingcan Li Machine Learning Lab, ECML-PKDD 2018 Presentation PROBLEM DEFINITION

372 views • 15 slides
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep

540 views • 33 slides
A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok

A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok

A Bayesian Approach to Generative Adversarial Imitation Learning NeurIPS 2018 Presenter Wonseok Jeon @ KAIST Joint work with Seokin Seo @ KAIST Kee-Eung Kim @ KAIST & PROWLER.io Imitation Learning A Markov decision process (MDP)

431 views • 14 slides
Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech

Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech

Fraunhofer Image Processing Heinrich Hertz Institute Interpretable & Transparent Deep Learning Fraunhofer HHI, Machine Learning Group Wojciech Samek Northern Lights Deep Learning Workshop (NLDL19) Troms, Norway 8th January 2019

1.15k views • 51 slides
Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent

Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent state reward action S t R t A t R t+ 1 Environment S t+ 1

1.07k views • 52 slides
Not Just a Black Box: Interpretable Deep Learning for Genomics Avan> Shrikumar, Peyton

Not Just a Black Box: Interpretable Deep Learning for Genomics Avan> Shrikumar, Peyton

Not Just a Black Box: Interpretable Deep Learning for Genomics Avan> Shrikumar, Peyton Greenside, Anshul Kundaje Peyton Anshul 1 With great power comes really poor interpretability Deep Interpretable Deep Power Learning Learning

783 views • 41 slides
10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018

10703 Deep Reinforcement Learning Imitation Learning - 1 Tom Mitchell November 4, 2018 Recommended readings: Used Materials Much of the material and slides for this lecture were borrowed from Katerina Fragkiadaki, and Ruslan

1.25k views • 49 slides
Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Table of Contents Behavioral Cloning 1 Inverse Reinforcement Learning 2 Apprenticeship

Lecture 7: Imitation Learning 2 Emma Brunskill CS234 Reinforcement Learning. Winter 2018 2 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning 3 Emma Brunskill (CS234 Reinforcement Learning. ) Winter 2018 1 /

1.27k views • 45 slides
Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2020 1 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill (CS234

526 views • 52 slides
Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Refresh Your Knowledge 6 Experience replay in deep Q-learning (select all): Involves using a bank

Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill CS234 Reinforcement Learning. Winter 2020 1 With slides from Katerina Fragkiadaki and Pieter Abbeel Lecture 7: Imitation Learning in Large State Spaces 1 Emma Brunskill (CS234

797 views • 57 slides

More recommend