Interactive Reinforcement Learning Human Generated Reward - PowerPoint PPT Presentation

Mar 12, 2024 •165 likes •252 views

Interactive Reinforcement Learning Human Generated Reward Presentation for Summer Camp 2015 May 25 2015 Reinforcement Learning Trial and error learning Explore and exploit Sutton and Barto 1988 Represent, predict and control

Interactive   Reinforcement Learning Human Generated Reward Presentation for Summer Camp 2015 May 25 2015
Reinforcement Learning • Trial and error learning • Explore and exploit Sutton and Barto 1988 • Represent, predict and control • Connect actions with rewards • Maximize future reward
Interactive Machine Learning Fails and Olsen Jr. 2003
Human Generated Reward • Humans know more! • Shaping systems to adapt Knox and Stone 2012 • Effectively reward learning • Transfer learning through collaboration • How can RL harness human reward?
Learning from Shaping Learning from Advice Kuhlmann et al. 2004 Blumberg et al. 2002 Learning from Demonstration Left: Argall et al. 2010   Right: Koenemann et al. 2014 Thomaz et al. 2006
Learning from Trial and Error Learning from Refinement Levine et al. 2015 Cakmak et al. 2012
Application • Shared control • Augmented representation • Integrate human and   non-human interaction • Autonomous prosthetics

Recommend

Reward Shaping in Episodic Reinforcement Learning Marek Grze s Canterbury, UK AAMAS 2017

Reward Shaping in Episodic Reinforcement Learning Marek Grze s Canterbury, UK AAMAS 2017 S ao Paulo, May 812 Motivating Reward Shaping Reinforcement Learning Agent reward state action r t a t s t r t+ 1 Environment s t+ 1 [Sutt

1.41k views • 16 slides

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays

Inverse Reinforcement Learning CS 294-112: Deep Reinforcement Learning Sergey Levine Todays Lecture 1. So far: manually design reward function to define a task 2. What if we want to learn the reward function from observing an expert, and

650 views • 33 slides

Foundations of Machine Learning Reinforcement Learning Reinforcement Learning Agent exploring

Foundations of Machine Learning Reinforcement Learning Reinforcement Learning Agent exploring environment. Interactions with environment: action state Agent Environment reward Problem: find action policy that maximizes cumulative reward

828 views • 66 slides

Reinforcement Learning: How Does It Work? We detect a state Reinforcement Learning We choose an

1 Reinforcement Learning: How Does It Work? We detect a state Reinforcement Learning We choose an action Lecture 2 We get a reward Gillian Hayes Our aim is to learn a policy what action to choose in what state to get maximum reward 11th

641 views • 5 slides

Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers (the class labels) Reinforcement learning: In reinforcement learning, the agent learns Markov Decision Processes based on a punishment/reward

89 views • 6 slides

Where's The Reward? Where's The Reward? A Review of Reinforcement Learning for Instructional

Where's The Reward? Where's The Reward? A Review of Reinforcement Learning for Instructional Sequencing Shayan Doroudi 1 2 2 Research Question Research Question Over the past 50 years, how Over the past 50 years, how successful has RL

1.92k views • 163 slides

Using Natural Language for Reward Shaping in Reinforcement Learning Prasoon Goyal , Scott Niekum

Using Natural Language for Reward Shaping in Reinforcement Learning Prasoon Goyal , Scott Niekum and Raymond J. Mooney The University of Texas at Austin Motivation 2 Motivation In sparse reward settings, random exploration has very

936 views • 47 slides

Learning frameworks Associative reinforcement learning Given input, learn to produce output

Learning frameworks Associative reinforcement learning Given input, learn to produce output (action) that maximizes immediate reward Supervised learning Modified Associative reward-penalty ( A R - P ) Assumes environment specifies correct output

241 views • 5 slides

1 Video of Demo Q-Learning Auto Cliff Grid Exploration vs. Exploitation How to Explore? Video

Reinforcement Learning Reinforcement Learning II We still assume an MDP: A set of states s S A set of actions (per state) A A model T(s,a,s) A reward function R(s,a,s) Still looking for a policy (s) New

358 views • 6 slides

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement

1 Deep Reinforcement Learning Qianqian Li, Nayeon Koong, Langtian He What is deep reinforcement learning? Agent/Actor + Action + Environment + State + Reward How does reinforcement learning work?

792 views • 31 slides

Reinforcement Learning with a Corrupted Reward Channel Tom Everitt, Victoria Krakovna, Laurent

Reinforcement Learning with a Corrupted Reward Channel Tom Everitt, Victoria Krakovna, Laurent Orseau, Marcus Hutter, Shane Legg Australian National University Google DeepMind IJCAI 17 and arXiv Motivation We will need to control

275 views • 11 slides

Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent

Carnegie Mellon School of Computer Science Deep Reinforcement Learning and Control Imitation Learning Spring 2019, CMU 10-403 Katerina Fragkiadaki Reinforcement learning Agent state reward action S t R t A t R t+ 1 Environment S t+ 1

1.07k views • 52 slides

Reinforcement Learning with a Corrupted Reward Channel Tom Everitt, Victoria Krakovna, Laurent

Reinforcement Learning with a Corrupted Reward Channel Tom Everitt, Victoria Krakovna, Laurent Orseau, Marcus Hutter, Shane Legg IJCAI 2017 and arXiv (slides adapted from Tom's IJCAI talk) Motivation Want to give RL agents good incentives

390 views • 20 slides

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature

Deep Reinforcement Learning [Human-Level Control through deep reinforcement learning, Nature 2015] CS 486/686 University of Waterloo Lecture 20: July 10, 2017 Outline Value Function Approximation Linear approximation Neural

706 views • 19 slides

Reinforcement Learning A (almost) quick (and very incomplete) introduction Slides from David

Reinforcement Learning A (almost) quick (and very incomplete) introduction Slides from David Silver, Dan Klein, Mausam, Dan Weld Reinforcement Learning At each time step t : Agent executes an action A t Environment emits a reward Rt

1.1k views • 71 slides

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning

Deep he(a)p, big feat arXiv:1707.06887 A Distributional Perspective on Reinforcement Learning arXiv:1702.08165 Reinforcement Learning with Deep Energy-Based Policies 1 / 25 Reinforcement Learning Environment Action Reward Interpreter State

531 views • 25 slides

Introduction to Reinforcement Learning Abdeslam Boularias Monday, June 25, 2018 1 / 93 What is

Machine Learning Summer School in Algiers Introduction to Reinforcement Learning Abdeslam Boularias Monday, June 25, 2018 1 / 93 What is reinforcement learning? a way of programming agents by reward and punishment without needing to specify

1.15k views • 93 slides

CSE-571 Probabilistic Robotics Passive: Policy given, transition model and reward are

Reinforcement Learning Same setting as MDP CSE-571 Probabilistic Robotics Passive: Policy given, transition model and reward are unknown Robot wants to learn value function for the given policy Reinforcement Learning Similar

212 views • 8 slides

Monte Carlo Approaches to Reinforcement Learning Robert Platt (w/ Marcus Gualtieris edits)

Monte Carlo Approaches to Reinforcement Learning Robert Platt (w/ Marcus Gualtieris edits) Northeastern University Model Free Reinforcement Learning Joystick command Agent World Observe screen pixels Reward = game score Goal: learn a

895 views • 47 slides

Lecture 14: Introduction to Reinforcement Learning CS109B Data Science 2 Pavlos Protopapas and

Lecture 14: Introduction to Reinforcement Learning CS109B Data Science 2 Pavlos Protopapas and Mark Glickman Outline What is Reinforcement Learning RL Formalism 1. Reward 2. The agent 3. The environment 4. Actions 5.

1.27k views • 64 slides

Reinforcement Learning l Variation on Supervised Learning l Exact target outputs are not given l

Reinforcement Learning l Variation on Supervised Learning l Exact target outputs are not given l Some variation of reward is given either immediately or after some steps Chess Path Discovery l RL systems learn a mapping from states to

793 views • 36 slides

AIXI: Universal Optimal Sequential Decision Making Marcus Hutter (2005) Reinforcement Learning

AIXI: Universal Optimal Sequential Decision Making Marcus Hutter (2005) Reinforcement Learning State space , Action space , Policy , Reward (, ) Goal: Find policy which maximizes expected cumulative reward.

552 views • 27 slides

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Introduction to Reinforcement Learning RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem Inside an RL agent Temporal difference learning Many faces of Reinforcement Learning What is

551 views • 35 slides

Reinforcement Learning Applied to RTS games Leonardo Rosa Amado Felipe Meneguzzi 8 May, 2017

Reinforcement Learning Applied to RTS games Leonardo Rosa Amado Felipe Meneguzzi 8 May, 2017 PUCRS Introduction Introduction Reinforcement learning focuses on maximizing the total reward of an agent through repeated interactions with an

759 views • 28 slides