Imitating Latent Policies from Observation Ashley D. Edwards, - PowerPoint PPT Presentation

Aug 09, 2023 •434 likes •553 views

Imitating Latent Policies from Observation Ashley D. Edwards, Himanshu Sahni, Yannick Schroecker, Charles L. Isbell Georgia Institute of Technology Introduction Imitation from Observation enables learning from state sequences Typical

Imitating Latent Policies from Observation Ashley D. Edwards, Himanshu Sahni, Yannick Schroecker, Charles L. Isbell Georgia Institute of Technology
Introduction • Imitation from Observation enables learning from state sequences • Typical approaches need extensive environment interactions • Humans can learn policies just by watching
Approach Given: Sequence of noisy expert observations Assumption: Discrete actions with deterministic transitions • z is defined as a latent action that caused a transition to occur • z can imply a real action or some other type of transition Action: Right Action: Right Z = 1 Z = 2 • A latent policy is the probability of taking a latent action in some state
Approach ILPO 1. Given sequence of observations, learn latent policy 2. Use a few environment steps to align actions
Approach ILPO 1. Given sequence of observations, learn latent policy 2. Use a few environment steps to align actions Latent policy network
Approach ILPO 1. Given sequence of observations, learn latent policy 2. Use a few environment steps to align actions (b) Action Remapping Network Action remapping network
Experiments: Classic Control • Access to expert observations only • No reward function used in approach • Comparison to Behavioral Cloning from Observation [1] [1] Torabi, Faraz, Garrett Warnell, and Peter Stone. "Behavioral cloning from observation." Proceedings of the 27th International Joint Conference on Artificial Intelligence . AAAI Press, 2018.
Experiments: CoinRun
Experiments: CoinRun
Thank You! Room: Pacific Ballroom at 6:30pm (Today)! Poster: #33

Recommend

Evaluating predictive loss for models with observation-level latent variables Russell Millar

Evaluating predictive loss for models with observation-level latent variables Russell Millar University of Auckland Dec 2015 Russell Millar University of Auckland Predictive loss Dec 2015 1 / 20 Motivation So, youve fitted a Bayesian

275 views • 26 slides

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides

C unobserved construct (e.g. Disordered v. Non- Disordered) Latent classes are mutually

University of Ulster at Magee, Friday 15 th June 2012 Latent Transition Analysis Overview of latent class and Dr Oliver Perra latent transition models Institute of Child Care Research Queens University Belfast email : o.perra@qub.ac.uk

812 views • 38 slides

Imitating Reality A Moulage Experience Laerdal SUN Conference October 2018 Orlando, FL About

Imitating Reality A Moulage Experience Laerdal SUN Conference October 2018 Orlando, FL About SIMETRI SIMETRI designs and develops creative technologies and capabilities for medical training. We are a woman- owned, minority-owned small

349 views • 31 slides

Qatar observation stations Qatar observation stations, Instruments and calibrations By By

Qatar observation stations Qatar observation stations, Instruments and calibrations By By Mohammed Almarri Qatar Meteorology Department 1 observation stations Number of surface observation Distribution map of stations Case examples

586 views • 34 slides

Recognizing and Imitating Programmer Style: Adversaries in Program Authorship Attribution Lucy

Recognizing and Imitating Programmer Style: Adversaries in Program Authorship Attribution Lucy Simko , Luke Zettlemoyer, Tadayoshi Kohno simkol@cs.washington.edu homes.cs.washington.edu/~simkol sim Source Code Attribution B int main() { A

565 views • 52 slides

A Rapidly Progressive Hydatid Disease Imitating Metastatic Malignancy: An Unusual Multi-Organ

Open Access SAJ Case Reports Case Report ISSN: 2375-7043 A Rapidly Progressive Hydatid Disease Imitating Metastatic Malignancy: An Unusual Multi-Organ Presentation. Report of a Case and Review of Literature Qays A. Hassan Al-Timimy * Section

338 views • 4 slides

Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent

Spectral Algorithms for Latent Variable Models Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent Variable Models, Edinburgh, UK Joint work with Mariya Ishteva, Ankur Parikh, Eric Xing, Byron Boots , Geoff

548 views • 40 slides

Observation of THz CSR Observation of THz CSR Burst at UVSOR- -II II Burst at UVSOR 1 Miho

Observation of THz CSR Observation of THz CSR Burst at UVSOR- -II II Burst at UVSOR 1 Miho SHIMADA, 1,2 Masahiro KATOH, 1* Akira MOCHIHASHI, 1 Shin-ichi KIMURA, 2 Masahito HOSAKA, 2 Yoshifumi TAKASHIMA, 3 Toshiharu TAKAHASHI 1 UVSOR Institute

282 views • 25 slides

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330 Student Presentation Table of Contents Motivation & problem Method overview Experiments Takeaways Discussion (strengths

653 views • 24 slides

Demystifying Relational Latent Representations Sebastijan Dumani, Hendrik Blockeel DTAI, KU

Demystifying Relational Latent Representations Sebastijan Dumani, Hendrik Blockeel DTAI, KU Leuven September 6, ILP 2017 1 Outline 2/24 1 Introduction 2 Understanding latent features 3 Properties of latent spaces Demystifying

281 views • 24 slides

The observation of Z c (4025) Landiao Liu ( ) PKU ( ) UCAS (

The observation of Z c (4025) Landiao Liu ( ) PKU ( ) UCAS ( ) BESIII Collaboration Outline Introduction The observation of Zc(4025) +/- The observation of Zc(4025) 0 Summary 2 PART 1

850 views • 36 slides

Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models

2/4/2020 Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models Expectation Maximization Algorithm (EM) Factor Analysis Probabilistic Principal Component Analysis Model Formulation Maximum

654 views • 18 slides

Movement Analysis Observation, Evaluation and Prescription Open Parallel Obs Observation -

Movement Analysis Observation, Evaluation and Prescription Open Parallel Obs Observation - n - Phase I hase I Desired Outcomes: SKI PERF SKI PERFORMANCE Consistent Speed Round turn shape Skis remain parallel Describe

231 views • 21 slides

Classroom observation Classroom observation Watch the impact not the teacher! 1100+

Classroom observation Classroom observation Watch the impact not the teacher! 1100+ meta-analysis 65,000 studies, and billion students and now adding affective outcomes No. of effects d=.40 Yes, there are high and low

372 views • 18 slides

Retrieval by Content Part 3: Text Retrieval Latent Semantic Indexing Srihari: CSE 626 1 Latent

Retrieval by Content Part 3: Text Retrieval Latent Semantic Indexing Srihari: CSE 626 1 Latent Semantic Indexing (LSI) Disadvantage of exclusive use of representing a document as a T-dimensional vector of term weights Users may pose

667 views • 15 slides

Observation from Space - GPM - GCOM-W SHIZUKU Misako KACHI Earth Observation Research

Observation from Space - GPM - GCOM-W SHIZUKU Misako KACHI Earth Observation Research Center (EORC) Japan Aerospace Exploration Agency (JAXA) The ALOS-2 satellite successfully launched on May 24, 2014! Plan to release first light 1

564 views • 20 slides

Optimization-Based Model Fitting for Latent Class and Latent Profile Analyses Guan-Hua Huang,

Optimization-Based Model Fitting for Latent Class and Latent Profile Analyses Guan-Hua Huang, Su-Mei Wang and Chung-Chu Hsu 12/17/2010 Breast cancer data van't Veer et al . Nature 2002 The 78 sporadic lymph-node-negative breast cancer

358 views • 24 slides

Report - BY CA M A H E N D R A SA N G H V I 2 Few samples of observation which one can

Clauses of Tax Audit Report - BY CA M A H E N D R A SA N G H V I 2 Few samples of observation which one can consider to clarify in Form 3CB are: Sr Qualification Type Observation/Qualification No. 1. Records produced for verification of

1.15k views • 89 slides

What is a latent v ariable ? SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R George Mo

What is a latent v ariable ? SU R VE Y AN D ME ASU R E ME N T D E VE L OP ME N T IN R George Mo u nt Data anal y tics ed u cator Meas u ring " lo y alt y" Is a " latent v ariable " re ected in the " Brand Lo y alt

580 views • 26 slides

Empirical Analysis of Latent Space Embedding David Mount and Eunhui Park Department of Computer

Latent Space Embeddings Optimization Exploration Tool Empirical Analysis of Latent Space Embedding David Mount and Eunhui Park Department of Computer Science University of Maryland, College Park MURI Meeting June 3, 2011 Latent Space

204 views • 17 slides

Latent Class Models: The Latent Class Logit Model Accouting for unobserved heterogeneity:

Latent Class Models: The Latent Class Logit Model Accouting for unobserved heterogeneity: Accouting for unobserved heterogeneity: Random Parameters Parametric assumption: must specify the functional form of the mixing distribution (for

190 views • 8 slides

Latent Damage and Reliability in Semiconductor Devices May1625 - Advisor & Client: Dr. Randy

Latent Damage and Reliability in Semiconductor Devices May1625 - Advisor & Client: Dr. Randy Geiger Sean Santella Hayle Olson David Ackerman Jaehyuk Han 04/27/16 Latent Damage and Reliability in Semiconductor Devices What is Latent

584 views • 40 slides

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides