Le Learning De Deep Co Control Po Policies fo for Au Autonomous - PowerPoint PPT Presentation

Dec 09, 2022 •291 likes •400 views

Le Learning De Deep Co Control Po Policies fo for Au Autonomous Ae Aerial Ve Vehicles wi with MP MPC- -Gu Guided Po Policy Se Search Tianhao Zhang, Gregory Kahn, Sergey Levine, Pieter Abbeel

Le Learning De Deep Co Control Po Policies fo for Au Autonomous Ae Aerial Ve Vehicles wi with MP MPC-‑ -‑Gu Guided Po Policy Se Search Tianhao ¡Zhang, ¡Gregory ¡Kahn, ¡Sergey ¡Levine, ¡Pieter ¡Abbeel Berkeley ¡Artificial ¡Intelligence ¡Research ¡Laboratory ¡(BAIR)
Mo Motivation • Enable ¡autonomous ¡aerial ¡vehicles ¡(AAVs) to ¡navigate ¡complex, ¡unstructured ¡environments
Ch Challenges • Complex, ¡unstructured ¡environments • no ¡explicit ¡state ¡estimation • Use ¡raw ¡observations ¡from ¡onboard ¡sensors Guided ¡Policy ¡Search ¡* ¡ • high ¡dimensionality ¡ and ¡non-‑linearity • Real-‑time ¡evaluation ¡at ¡test ¡time • computationally ¡efficient • Robust ¡to ¡model ¡errors ¡and ¡environment ¡ disturbances Model ¡Predictive ¡Control • flying ¡systems ¡are ¡prone ¡to ¡catastrophic ¡failures * ¡S. ¡Levine ¡& ¡P . ¡Abbeel. ¡"Learning ¡neural ¡network ¡policies ¡with ¡guided ¡policy ¡search ¡under ¡unknown ¡dynamics." NIPS . ¡2014. S. ¡Levine ¡et ¡al. ¡"End-‑to-‑end ¡training ¡of ¡deep ¡visuomotor policies." JMLR. 2015.
Ap Approach: ¡ ¡MPC-‑ -‑GP GPS • Guided ¡Policy ¡Search ¡(GPS) Objective ¡Cost • Trajectory ¡optimization • Supervised ¡ learning • Policy ¡agreement Policy ¡ Trajectory ¡ Online ¡MPC Agreement ¡Cost • MPC-‑GPS Optimization • Substitute ¡offline ¡trajectory ¡ optimization ¡for ¡online ¡MPC Execute on ¡Robot Train ¡Neural ¡ Network
Ap Approach: ¡ ¡MPC-‑ -‑GP GPS ¡ ¡(cont.) • Training: ¡ • use ¡instrumented ¡setup ¡to ¡obtain ¡ full ¡state ¡information ¡ x • MPC ¡uses ¡ x to ¡generate ¡trajectories • Record ¡observations ¡ o • Policy ¡is ¡trained ¡to ¡map ¡from ¡ o to ¡ u • Test: • No ¡need ¡for ¡instrumented ¡ setup • Policy ¡runs ¡in ¡closed ¡loop
Experiment ¡ Ex ¡Ov Overview Hallway Cylinder Training ¡ Environments • no ¡model ¡error • 0.05kg mass ¡error Model ¡Errors 8% rotor ¡bias perturbed ¡ model ¡params • •
Ex Experimental ¡ ¡Evaluation Cylinder ¡-‑ 0.05kg ¡mass ¡error Baseline MPC-‑GPS
Ex Experimental ¡ ¡Ev Evaluation ¡ ¡(cont.) 2x
Ex Experimental ¡ ¡Evaluation ¡ ¡(cont.) 3x
Thank ¡you!

Recommend

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature 2015] CS 486/686 University of Waterloo Lecture 20: July 10, 2017 Outline Value Function Approximation Linear approximation Neural

706 views • 19 slides

Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted

Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewards Umer Siddique , Paul Weng, and Matthieu Zimmer University of Michigan-Shanghai Jiao Tong University Joint Institute ICML 2020 U. Siddique,

516 views • 28 slides

Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Introduction to Deep Reinforcement Learning and Control Spring 2019, CMU 10-403 Katerina Fragkiadaki Course Logistics Course website : all you need to

1.13k views • 88 slides

DeepMind Self-Learning Atari Agent Human - level control through deep reinforcement learning

DeepMind Self-Learning Atari Agent Human - level control through deep reinforcement learning Nature Vol 518, Feb 26, 2015 The Deep Mind of Demis Hassabis Backchannel / Medium.com interview with David Levy Advanced Topics:

989 views • 68 slides

Deep Q Learning CMU 10-403 Katerina Fragkiadaki Used Materials Disclaimer : Much of the

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Deep Q Learning CMU 10-403 Katerina Fragkiadaki Used Materials Disclaimer : Much of the material and slides for this lecture were borrowed from Russ

1.09k views • 64 slides

Deep Learning on GPU Mattias Flt Dept. of Automatic Control Lund Institute of Technology

Deep Learning on GPU Mattias Flt Dept. of Automatic Control Lund Institute of Technology Mattias Flt Deep Learning on GPU Overview What is the difference between CPU and GPU? What is CUDA, and how does it relate to cuBLAS and cuDNN? How

329 views • 13 slides

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain

Model-Free Control (Reinforcement Learning) and Deep Learning M ARC G. B ELLEMARE Google Brain (Montral) 1956 1992 2016 2015 T HE A RCADE L EARNING E NVIRONMENT (B ELLEMARE ET AL ., 2013) 160 pixels Reward: change in score 210 pixels

793 views • 55 slides

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides

Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms

Deep 3D Representation Learning for Visual Computing Hao Su July 6, 2017 Outline Overview of 3D deep learning 3D deep learning algorithms Conclusion 2 Outline Overview of 3D deep learning Background 3D deep learning tasks 3D deep

1.66k views • 122 slides

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP

ACCELERATE DEEP LEARNING WITH NVIDIA'S DEEP LEARNING PLATFORM | STEPHEN JONES | GTC16 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY MEDIA & ENTERTAINMENT SECURITY & DEFENSE AUTONOMOUS MACHINES Image

311 views • 26 slides

Reinforcement Learning Stephen D. Scott (Adapted from Tom Mitchells slides) 1 Outline

Reinforcement Learning Stephen D. Scott (Adapted from Tom Mitchells slides) 1 Outline Control learning Control policies that choose optimal actions Q learning Convergence Temporal difference learning 2 Control Learning

576 views • 20 slides

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice

Deep Learning on GPUs March 2016 What is Deep Learning? GPUs and DL AGENDA DL in practice Scaling up DL 2 What is Deep Learning? 3 DEEP LEARNING EVERYWHERE INTERNET & CLOUD MEDICINE & BIOLOGY SECURITY & DEFENSE MEDIA &

1.45k views • 39 slides

Visual Learning with Unlabeled Video and Look-Around Policies Kristen Grauman Department of

Visual Learning with Unlabeled Video and Look-Around Policies Kristen Grauman Department of Computer Science University of Texas at Austin Visual recognition: significant recent progress Big labeled Deep learning datasets ImageNet top-5

804 views • 52 slides

Continuous Control With Deep Reinforcement Learning Timothy P. Lillicrap , Jonathan J. Hunt

Continuous Control With Deep Reinforcement Learning Timothy P. Lillicrap , Jonathan J. Hunt , Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver & Daan Wierstra Presenters: Anqi (Joyce) Yang Jonah Philion Jan 21

1.64k views • 101 slides

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning

Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods) Peter Henderson Statistical Society of Canada Annual Meeting 2018 Contributors Riashat Islam Phil Bachman Doina Precup David Meger Joelle

850 views • 37 slides

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI

Image Classification with DIGITS NVIDIA Deep Learning Institute 1 DEEP LEARNING INSTITUTE DLI Mission Helping people solve challenging problems using AI and deep learning. Developers, data scientists and engineers Self-driving cars,

1.44k views • 72 slides

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin

Natural Language Processing with Deep Learning CS224N The Future of Deep Learning + NLP Kevin Clark Deep Learning for NLP 5 years ago No Seq2Seq No Attention No large-scale QA/reading comprehension datasets No TensorFlow or

1.41k views • 92 slides

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning

An Overview of Deep Residual Learning Semih Yagcioglu 01.03.2016 Deep Residual Learning Microsoft Research Asia (MSRA) Kaiming He, Xiangyu Zhang, Shaoqing Ren, & Jian Sun. Deep Residual Learning for Image Recognition. arXiv

1.32k views • 54 slides

Exploration: Part 2 CS 285: Deep Reinforcement Learning, Decision Making, and Control Sergey

Exploration: Part 2 CS 285: Deep Reinforcement Learning, Decision Making, and Control Sergey Levine Class Notes 1. Homework 4 due today! Recap: whats the problem? this is easy (mostly) this is impossible Why? Recap: classes of

587 views • 39 slides

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

541 views • 15 slides

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning

Mocha.jl Deep Learning in Julia Chiyuan Zhang (@pluskid) CSAIL, MIT Deep Learning Learning with multi-layer (3~30) neural networks, on a huge training set. State-of-the-art on many AI tasks Computer Vision : Image Classification,

1.53k views • 23 slides

Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL

Tofu: Parallelizing Deep Learning Systems with Automatic Tiling Minjie Wang Deep Learning Deep Learning trend in the past 10 years Caffe State-of-art DL system is based on dataflow GPU#0 w1 w2 data g1 g2 Forward propagation

405 views • 38 slides

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides