Learning Latent Dynamics for Planning from Pixels Danijar Hafner, - PowerPoint PPT Presentation

Jul 03, 2023 •223 likes •446 views

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijar h danijar.com/planet Planning with Learned Models Watter et al., 2015, Banijamali

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijar h danijar.com/planet
Planning with Learned Models Watter et al., 2015, Banijamali et al. 2017, Zhang et al. 2017 Agrawal et al., 2016; Finn & Levine, 2016; Ebert et al., 2018
Visual Control Tasks partially many sparse contacts balance observable joints reward Some model-free methods can solve these tasks but need up to 100,000 episodes
Visual Control Tasks partially many sparse contacts balance observable joints reward Some model-free methods can solve these tasks but need up to 100,000 episodes
We introduce PlaNet Recipe for scalable model-based reinforcement learning 1 Efficient planning in latent space with large batch size 2 Reaches top performance using 200X fewer episodes 3
Latent Dynamics Model encode images
Latent Dynamics Model encode images predict states
Latent Dynamics Model encode images predict states decode images
Latent Dynamics Model encode images predict states decode images decode rewards
Recurrent State Space Model deterministic stochastic h 1 h 2 h 3 h 1 h 2 h 3 z 1 s 1 z 2 s 2 z 3 s 3 z 1 z 2 z 3 Recurrent Neural Network State Space Model Recurrent State Space Model
Unguided Video Predictions by Single Agent 5 frames context and 45 frames predicted
Unguided Video Predictions by Single Agent 5 frames context and 45 frames predicted
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Planning in Latent Space
Comparison to Model-Free Agents Training time 1 day on a single GPU
Enabling More Model-Based RL Research Explore dynamics Distill the planner to save Value function to extend without supervision computation planning horizon
Learning Latent Dynamics for Planning from Pixels Website with code, videos, blog post, animated paper: danijar.com/planet 33

Recommend

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer,

Learning Latent Dynamics for Planning from Pixels Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, James Davidson @danijarh danijar.com/planet Planning with Learned Models Watter et al., 2015, Banijamali

519 views • 31 slides

Deep Learning Helicopter Dynamics Models Ali Punjani Pieter Abbeel UC Berkeley EECS Latent

Deep Learning Helicopter Dynamics Models Ali Punjani Pieter Abbeel UC Berkeley EECS Latent State: Airflow, Flexibility, Engine Dynamics etc. Similar trajectories have similar dynamics acceler acceleration ation stat state-contr e-control

372 views • 12 slides

Dream to Control Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap,

Dream to Control Learning Behaviors by Latent Imagination Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi Google Brain DeepMind @danijarh danijar.com/dreamer We introduce Dreamer Scalable reinforcement learning from pixels

428 views • 30 slides

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330

Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model CS330 Student Presentation Table of Contents Motivation & problem Method overview Experiments Takeaways Discussion (strengths

653 views • 24 slides

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare Simple Representations for RL 2 12 DeepMDP Latent Space Model: Neural networks MDP:

573 views • 17 slides

Pixels Pixels Row and column indicates a PIXEL not a POINT. A pixel can theoretically contain

Pixels Pixels Row and column indicates a PIXEL not a POINT. A pixel can theoretically contain infinitely many points Drawing lines with pixels: fill pixels along the trajectory of a line from point p1 to p2 Aliasing Antialiasing

498 views • 15 slides

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides

La Latent-sp space Dynam Dynamics ics for r Re Reduced Deformable Simulation Lawson Fulton

La Latent-sp space Dynam Dynamics ics for r Re Reduced Deformable Simulation Lawson Fulton 1,2 , Vismay Modi 1 , David Duvenaud 1 , David I.W. Levin 1 , Alec Jacobson 1 1 University of Toronto, Canada 2 MESH Consultants, Canada The 40

772 views • 59 slides

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain (Montral) 1956 1992 2016 2015 T HE A RCADE L EARNING E NVIRONMENT (B ELLEMARE ET AL ., 2013) 160 pixels Reward: change in score 210 pixels

793 views • 55 slides

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in machine learning? Training continuous latent- variable models (VAEs, GANs) to produce large images Training large supervised models with

1.26k views • 61 slides

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine Joint work with Anima Anandkumar Rong Ge UC Irvine Microsoft Research Latent Variable Modeling Goal: Discover hidden effects from observed

1.27k views • 115 slides

Decoding in Latent Conditional Models: A Practically Fast Solution for an NP-hard Problem Xu Sun (

Latent dynamics workshop 2010 Decoding in Latent Conditional Models: A Practically Fast Solution for an NP-hard Problem Xu Sun ( ) University of Tokyo 2010.06.16 Outline Introduction Related Work & Motivations Our

741 views • 49 slides

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of Maryland furongh@cs.umd.edu ACM SIGMETRICS Tutorial 2018 1 / 75 Tutorial Topic Learning algorithms for latent variable models based on

1.96k views • 180 slides

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides

? Which intermediate 4 pixels to turn on? 3 (3,2) 2 1 0 1 2 3 4 5 6 7 8 9 10 11

Recall: Line drawing algorithm Programmer specifies (x,y) of end pixels Need algorithm to determine pixels on line path 8 Line: (3,2) -> (9,6) 7 (9,6) 6 5 ? Which intermediate 4 pixels to turn on? 3 (3,2) 2 1 0 1 2 3 4

374 views • 34 slides

Learning Structural SVMs with Latent Variables Presented By- Subhabrata Debnath(Roll-

Learning Structural SVMs with Latent Variables Presented By- Subhabrata Debnath(Roll- 13111063) Anjan Banerjee(Roll-13111008) Basics Machine Learning Blood Sugar Hyper Pressure Tension 10 15 No 5 5 No 25 25 Yes 30 36

795 views • 38 slides

Learning with Latent Language Jacob Andreas, Dan Klein, Sergey Levine CS330 Student Presentation

Learning with Latent Language Jacob Andreas, Dan Klein, Sergey Levine CS330 Student Presentation Motivation The structure of natural language reflects the structure of the world. The authors propose to use language as a latent parameter space

838 views • 21 slides

Chapter 4: Modifying Pixels in a Range Reminder: Pixels are in a matrix Matrices have two

Chapter 4: Modifying Pixels in a Range Reminder: Pixels are in a matrix Matrices have two dimensions: A height and a width We can reference any element in the matrix with (x,y) or (horizontal, vertical) We refer to those coordinates as

915 views • 75 slides

Reinforcement learning with raw image pixels as input state Damien Ernst , Rapha ee , Louis

Reinforcement learning with raw image pixels as input state Damien Ernst , Rapha ee , Louis Wehenkel el Mar Department of Electrical Engineering and Computer Science University of Li` ege, Belgium Postdoctoral Researcher FNRS

741 views • 14 slides

C unobserved construct (e.g. Disordered v. Non- Disordered) Latent classes are mutually

University of Ulster at Magee, Friday 15 th June 2012 Latent Transition Analysis Overview of latent class and Dr Oliver Perra latent transition models Institute of Child Care Research Queens University Belfast email : o.perra@qub.ac.uk

812 views • 38 slides

Unsupervised learning: latent space analysis and clustering Yifeng Tao School of Computer

Introduction to Machine Learning Unsupervised learning: latent space analysis and clustering Yifeng Tao School of Computer Science Carnegie Mellon University Slides adapted from Tom Mitchell, David Sontag, Ziv Bar-Joseph Yifeng Tao Carnegie

931 views • 62 slides

Learning Flat Latent Manifolds with VAEs Nutan Chen 1 , Alexej Klushyn 1 , Francesco Ferroni 2 ,

Learning Flat Latent Manifolds with VAEs Nutan Chen 1 , Alexej Klushyn 1 , Francesco Ferroni 2 , Justin Bayer 1 , Patrick van der Smagt 1 1 Machine Learning Research Lab, Volkswagen Group, Munich, Germany 2 Autonomous Intelligent Driving GmbH,

324 views • 17 slides