Reinforcement Learning in Psychology and Neuroscience with thanks - PowerPoint PPT Presentation

Reinforcement Learning in Psychology and Neuroscience with thanks to Elliot Ludvig Princeton University

Psychology has identified two primitive kinds of learning • Classical Conditioning • Operant Conditioning (a.k.a. Instrumental learning) • Computational theory: ❖ Classical = Prediction - What is going to happen? ❖ Operant = Control - What to do to maximize reward?

Classical Conditioning

Pavlov • Russian physiologist • Interested in how learning happened in the brain • Conditional and Unconditional Stimuli

Rescorla-Wagner Model (1972) • Computational model of conditioning ❖ Widely cited and used • Learning as violation of expectations ❖ TD learning as extension of RW

Operant Learning • Operant Conditioning is all about choice in 3 main ways: ❖ Decide which response to make? ❖ Decide how much to respond? ❖ Decide when to respond?

Thorndike’s Puzzle Box

Operant Chambers

Complex Cognition

Marr’s 3 Levels of Analysis • Computational ❖ What function is being fulfilled? • Algorithmic ❖ How is it accomplished? • Implementational ❖ What physical substrate is involved?

The Basic TD Model • Learn to predict discounted sum of upcoming reward through TD with linear function approximation: n � V t = w T t x t = w t ( i ) x t ( i ) i =1 • The TD error is calculated as: δ t = r t +1 + γ V t +1 − V t .

TD( λ ) algorithm/model/neuron Reward ∑ w i ⋅ x i States x i e i w i δ i or TD Features Value of state Error or action λ w i ~ δ ⋅ e i ˙ TD Eligibility Error Trace

Brain reward systems What signal does this neuron carry? Honeybee Brain VUM Neuron Hammer, Menzel

Dopamine • Small-molecule Neurotransmitter ❖ Diffuse projections from mid-brain throughout the brain from Pinel (2000), p.364 Key Idea: Phasic change in baseline dopamine responding = reward prediction error

Dopamine neurons signal the TD error error/change in prediction of reward Wolfram Schultz, et al.

Reward Unexpected Reward Value Representation- TD error independent Reward Expected predictions Cue of TD errors Value TD error Reward Absent Value TD error t = r t +1 + γ V t +1 � V t TD error

The theory that Dopamine = TD error is one of the most important interactions ever between artificial intelligence and neuroscience

Reinforcement Learning in Psychology and Neuroscience with thanks - PowerPoint PPT Presentation

Reinforcement Learning in Psychology and Neuroscience with thanks to Elliot Ludvig Princeton University Psychology has identified two primitive kinds of learning Classical Conditioning Operant Conditioning (a.k.a. Instrumental

Reinforcement Learning in Psychology and Neuroscience with thanks to Elliot Ludvig University

Reinforcement Learning AIMA Chapters: 21.1, 21.2, 21.3. Sutton and Barto, Reinforcement Learning:

Reinforcement Learning Timothy Chou Charlie Tong Vincent Zhuang April 19, 2016 Reinforcement

PSYCHOLOGY Dr Amy Pearson Lecturer in Psychology The Truth About Psychology Psychology

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Reinforcement Learning and Simulation-Based Search David Silver Reinforcement Learning and

Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning?

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Safe Reinforcement Learning Philip S. Thomas Stanford CS234: Reinforcement Learning, Guest

reinforcement learning in humans and animals nathaniel daw nyu neuroscience; psychology;

What is Health Psychology? Division of Health Psychology Scotland Psychology Focuses on

LGA 504 Sport Psychology Session Objectives Define Sport Psychology and examine why it is

Cognitive computational neuroscience of vision Nikolaus Kriegeskorte Department of Psychology,

Introduction to Reinforcement Learning Kevin Chen and Zack Khan Lecture 1: Introduction to

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

Biology and Psychology: Gateway to Neuroscience Deb Bidwell, M.S., Department of Biology, Mark

ADHD DHD in Primar imary y Care Andres J. Pumariega, M.D. Professor and Chair, Department of

Neuroscience of Decision Making Sam uel McClure Psycholog y Departm ent 1 5 May 2 0 0 8 The

European Bioinformatics Institute European Bioinformatics Institute British outstation of the

DEEP BRAIN STIMULATION: BEST PRACTICE AND MORE Francesca Morgante Institute of Molecular and

Disclosures I have no disclosures to make Doppler in Obstetric Management: How to Interpret

I have nothing to I have nothing to disclose disclose UC UC SF SF University of California

Intro to Ultrasound MSK September 19, 2020 DMUCOM Thomas Benzoni, DO, AOBEM, FACEP

Course Projects Class 9. 22 Sep 2009 Administrivia THURSDAYS CLASS: WEAN HALL 5403 n

Explore More Topics

Sambuz

Useful Links

Newsletter

Mail Us