Path following with reinforcement learning for autonomous cars - - PowerPoint PPT Presentation

Sep 11, 2023 •463 likes •714 views

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS) Index Basics of Reinforcement Learning Model Based vs Model Free Reinforcement Learning Autonomous Car collision avoidance What is Reinforcement

Path following with reinforcement learning for autonomous cars - Mozzam Motiwala (IAS)
Index ● Basics of Reinforcement Learning ● Model Based vs Model Free Reinforcement Learning ● Autonomous Car collision avoidance
What is Reinforcement Learning? ● Learning by trial and error only based on a reward signal[1] https://towardsdatascience.com/solving-the-multi-armed- bandit-problem-b72de40db97c Exploration vs Exploitation?
Markov-Desicion Process [1] Reward Function? Policy? Optimal Policy? Transition Function?
Some terminalogy ● Value Function: ● Action Value Function: Why Discounting Factor?
Gridworld [1]
Finding Optimal Policy [1]
Cart Pole Balancing Problem https://towardsdatascience.com/cartpole-introduction-to- reinforcement-learning-ed0eb5b58288 https://www.youtube.com/watch?v=Lt-KLtkDlh8
Index ● Basics of Reinforcement learning ● Model Based vs Model Free Reinforcement Learning ● Autonomous Car collision avoidance
Model-based By a model of the environment we mean anything that an agent can use to predict how the environment will respond to its actions[2]. https://towardsdatascience.com/model-based-reinforcement-learning-cb9e41ff1f0d
Example https://towardsdatascience.com/model-based-reinforcement-learning-cb9e41ff1f0d Whats Next? :: Now lets sample from it to adjust the policy..
Why model-based RL? Reduced number of Advantages? interaction with the real ● Fast environment while ● Need less data learning. Problems? Types: Neural Network Model, ● What if the model is wrong? Guassian Process Model.. etc
Model Based+ Model Free [2]
Results [1]
Why better result? [1]
Index ● Basics of Reinforcement learning ● Model Based vs Model Free Reinforcement Learning ● Autonomous Car Collision Avoidance
Application: Autonomous Car Why Reinforcement Learning? Problem with traditional methods ● Slow ● Assumptions Learning in RL ● Adapting to environment ● Learning from mistakes
Generalized Computation Graph Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs(GCG) for Robot Navigation[3] [3] ● H=1 : Model-Free ● H= N (Length of Episode): Model-Based
Model Details ● Deep RNN as Model ● Model output 1= Current Reward ŷ: Robots speed ● Model output 2= Future Value to go(value of the state) ^b: Distance travelled before collision ● Policy Evaluation Function : ● Policy Evaluation by sampling k random action sequence and selecting the one with max reward.
GCG : Algorithm [3]
Evaluation and Results https://www.youtube.com/watch?v=NlFbLVG6LpA [3]
Summary ● Benefits of Reinforcement Learning ● Model-Free vs Model-Based ● Combined approach that subsumes Model-free and Model-based
References 1. R. Sutton and A. Barto, Reinforcement Learning: An Introduction 2. R. Sutton, “Dyna, an Integrated Architecture for Learning, Planning,and Reacting,” in AAAI, 1991. 3. G. Kahn, A. Villaflor, B. Ding, P. Abbeel, and S. Levine. Self- Supervised Deep ReinforcementLearning with Generalized Computation Graphs for Robot Navigation. InIEEE InternationalConference on Robotics and Automation, 2018.
Question?

Recommend

Visual Reinforcement Learning with Imagined Goals Ashvin Nair, Vitchyr Pong , Murtaza Dalal,

Visual Reinforcement Learning with Imagined Goals Ashvin Nair*, Vitchyr Pong* , Murtaza Dalal, Shikhar Bahl, Steven Lin, Sergey Levine Autonomous Learning Autonomous Learning Autonomous Learning Reinforcement Learning? Autonomous Learning:

594 views • 21 slides

Advanced planning for autonomous vehicles using reinforcement learning and deep inverse

Advanced planning for autonomous vehicles using reinforcement learning and deep inverse reinforcement learning You, Lu, Filev, Tsiotras Laura Graves Overview Implements planning for autonomous vehicles using RL and inverse RL systems. This is

347 views • 14 slides

Reinforcement Learning for Safe Decision-Making in Autonomous Driving Edouard Leurent 1,2,3 ,

Reinforcement Learning for Safe Decision-Making in Autonomous Driving Edouard Leurent 1,2,3 , Odalric-Ambrym Maillard 1 , Denis Efimov 2 1 Inria SequeL, 2 Inria Valse, 3 Renault Group 01 Motivation and Scope 2 -Reinforcement Learning for

1.84k views • 139 slides

Autonomous Navigation CSE 571 Inverse Optimal Control (Inverse Reinforcement Learning) Many

5/25/20 Autonomous Navigation CSE 571 Inverse Optimal Control (Inverse Reinforcement Learning) Many slides by Drew Bagnell Carnegie Mellon University 1 2 Optimal Control Solution X X Learning Y Y Cost Map (Sensor Data) (Input)

489 views • 10 slides

Autonomous(Task(Sequencing(for(Customized( Curriculum(Design(in(Reinforcement(Learning

Autonomous(Task(Sequencing(for(Customized( Curriculum(Design(in(Reinforcement(Learning Sanmit'Narvekar, Jivko Sinapov,+and+Peter+Stone Department+of+Computer+Science University+of+Texas+at+Austin {sanmit,+jsinapov,+pstone}+@cs.utexas.edu

620 views • 26 slides

for Knowledge Transfer in Reinforcement Learning Benjamin Rosman Mobile Intelligent Autonomous

Structured Representations for Knowledge Transfer in Reinforcement Learning Benjamin Rosman Mobile Intelligent Autonomous Systems Council for Scientific and Industrial Research & School of Computer Science and Applied Maths University of

689 views • 52 slides

LunarLander-v2 using Deep Reinforcement Learning A project developed for Autonomous Agents

LunarLander-v2 using Deep Reinforcement Learning A project developed for Autonomous Agents Course PLH513 Portokalakis Petros February 2020 Simple Game 8-Dimensional state space 4 actions per state +100 points for landing

301 views • 10 slides

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Introduction to Reinforcement Learning RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem Inside an RL agent Temporal difference learning Many faces of Reinforcement Learning What is

551 views • 35 slides

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine playing a new game whose rules you dont know; after a hundred or so moves your don t know; after a hundred or so moves, your opponent announces, You

511 views • 30 slides

Safe Reinforcement Learning for Decision-Making in Autonomous Driving Edouard Leurent,

Safe Reinforcement Learning for Decision-Making in Autonomous Driving Edouard Leurent, Odalric-Ambrym Maillard, Denis Efimov, Wilfrid Perruquetti, Yann Blanco SequeL, Inria Lille Nord Europe Valse, Inria Lille Nord Europe Renault Group

978 views • 61 slides

Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers (the class labels) Reinforcement learning: In reinforcement learning, the agent learns Markov Decision Processes based on a punishment/reward

89 views • 6 slides

Reinforcement Learning l Variation on Supervised Learning l Exact target outputs are not given l

Reinforcement Learning l Variation on Supervised Learning l Exact target outputs are not given l Some variation of reward is given either immediately or after some steps Chess Path Discovery l RL systems learn a mapping from states to

793 views • 36 slides

Examples of Reinforcement Learning Robocup Soccer Teams Stone & Veloso, Reidmiller et al.

Reinforcement Learning Slides by Rich Sutton Mods by Dan Lizotte Refer to Reinforcement Learning: An Introduction by Sutton and Barto Alpaydin Chapter 16 Up until now we have been Supervised Learning Classifying, mostly

891 views • 31 slides

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning Haarnoja, Tang et al. (2017) Reinforcement Learning with Deep Energy Based Policies, ICML . Haarnoja, Zhou et al. (2018) Soft Actor-Critic: Off-Policy

683 views • 24 slides

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control Theory, you actually have some knowledge in RL. RL shares the overall goal with OCT: solving for a control policy such that the cumulative cost

610 views • 22 slides

AUTONOMOUS DRONE NAVIGATION WITH DEEP LEARNING Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith

AUTONOMOUS DRONE NAVIGATION WITH DEEP LEARNING Nikolai Smolyanskiy, Alexey Kamenev, Jeffrey Smith Project Redtail May 8, 2017 100% AUTONOMOUS FLIGHT OVER 1 KM FOREST TRAIL AT 3 M/S 2 Why autonomous path navigation? Our deep learning approach

394 views • 35 slides

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still assume a Marko kov decision process (MDP): A se set o of st states s s S A se set o of a actions ( s (per st state) A A mo model

96 views • 5 slides

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec 03 2008 1 Large State Spaces h When a problem has a large state space we can not longer represent the U or Q functions as explicit tables explicit

436 views • 18 slides

Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning?

Reinforcement Learning<br/><br/> 4/25/19, 8*06 PM Reinforcement Learning<br/><br/> 4/25/19, 8*06 PM Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning? Spring 2019 Created:

371 views • 15 slides

7. Motor Control and Reinforcement Learning Outline A. Action Selection and Reinforcement B.

7. Motor Control and Reinforcement Learning Outline A. Action Selection and Reinforcement B. Temporal Difference Reinforcement Learning C. PVLV Model D. Cerebellum and Error-driven Learning 2/23/18 COSC 494/594 CCN 2 Sensory-Motor Loop

790 views • 56 slides

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

541 views • 15 slides

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning Ja Jan-Wi

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning Ja Jan-Wi Willem van de Meent Northeastern University [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188

145 views • 10 slides

Autonomous Intelligent Robotics Instructor: Shiqi Zhang

Spring 2018 CIS 693, EEC 693, EEC 793: Autonomous Intelligent Robotics Instructor: Shiqi Zhang http://eecs.csuohio.edu/~szhang/teaching/18spring/ Reinforcement Learning Adapted form Peter Bodk Previous Lectures l Supervised learning l

990 views • 44 slides

W ISE M OVE ? A research platform that mimics our autonomous driving stack. Objective:

W ISE M OVE : A Framework to Investigate Safe Deep Reinforcement Learning for Autonomous Driving Jaeyoung Lee , Aravind Balakrishnan, Ashish Gaurav, Krzysztof Czarnecki, Sean Sedwards University of Waterloo September, 14th, 2019 1 W ISE M

186 views • 18 slides