A Neural Mechanism for Decision Making K C Y W D K D O P E D B A I Q - PowerPoint PPT Presentation

A Neural Mechanism for Decision Making K C Y W D K D O P E D B A I Q S D F M K C N F A E O I E N C V N S E N C H P D N C O E N A S H Q E N D N C K R N D N Q I O M Z C P Q

What is a decision? • A commitment to a proposition or selection of an action • Based on – evidence – prior knowledge – payoff

Why study decisions? • They are a model of higher brain function • They are experimentally tractable – Combined behavior and physiology in rhesus monkeys

Direction-Discrimination Task Reaction-time version

Direction-Discrimination Task Direction-Discrimination Task Reaction-time version

Direction-Discrimination Task Direction-Discrimination Task Reaction-time version

Direction-Discrimination Task Direction-Discrimination Task Reaction-time version

Direction-Discrimination Task Direction-Discrimination Task Reaction-time version Reward for correct choice

Psychometric function: Accuracy

Chronometric function: Speed Reaction time [ms]

Information is coded by spikes

Hubel, 1988 “Eye, brain and vision”

Sensory “Evidence”

Albright et al., 1984 J. Neurophysiol.

Spatially-selective, eye movement-related, persistent activity in area LIP 120 60 0 120 Spikes/sec 60 0 100 ms 60

LIP activity during direction discrimination task

LIP activity during direction discrimination task

LIP activity during direction discrimination task

Average LIP activity in RT motion task choose T in choose T out Roitman & Shadlen, 2002 J. Neurosci.

A Neural Integrator for Decisions? MT: Sensory Evidence LIP: Decision Formation Motion energy Accumulation of evidence “step” “ramp” High motion strength High motion strength Spikes/s Spikes/s h t g n e r t s n o Threshold Low motion strength i t o m w o L Time Time Stimulus Stimulus Stimulus Stimulus ~1 sec ~1 sec on off on off

Diffusion to bound model Positive bound Negative bound

Diffusion to bound model Positive bound or Criterion to answer “1” Negative bound or Criterion to answer “2” Proposed by Wald, 1947 and Turing (WW II, classified); Stone, 1960; then Laming, Link, Ratcliff, Smith, . . .

Diffusion to bound model Criterion to answer “Right” Accumulated evidence for Rightward Momentary evidence and e .g., against Leftward ∆ Spike rate: µ = kC MT Right – MT Left Criterion to answer “Left” C is motion strength (coherence) Shadlen & Gold (2004) Palmer et al (in press)

Best fitting chronometric function “Diffusion to bound” t ( C ) = B kC tanh( BkC ) + t nd

Predicted psychometric function “Diffusion to bound” 1 P = � 2 k C B 1 + e

Criterion to answer “Right” Accumulated evidence for Rightward Momentary evidence and e .g., against Leftward ∆ Spike rate: µ = kC MT Right – MT Left Criterion to answer “Left” Time (ms)

Criterion to answer “Right” Accumulated evidence for Rightward Momentary evidence and e .g., against Leftward ∆ Spike rate: µ = kC MT Right – MT Left Criterion to answer “Left” LIP represents ∫ dt of momentary motion evidence • • Momentary evidence is a spike rate difference from area MT • The accumulated evidence used by the monkey is in area LIP • How and where is the integral computed? • How is the bound set? • How is a bound crossing detected?

The momentary evidence is a ∆ between opposite direction signals in area MT Bound for RIGHT choice • More RIGHT choices Stimulate RIGHTWARD MT neurons • Faster RIGHT choices • Slower LEFT choices Bound for LEFT choice The accumulated evidence used by the monkey is in area LIP Bound for RIGHT choice • More RIGHT choices Stimulate RIGHT CHOICE LIP neurons • Faster RIGHT choices • Slower LEFT choices Bound for LEFT choice

Criterion to answer “Right” Accumulated evidence for Rightward Momentary evidence and e .g., against Leftward ∆ Spike rate: µ = kC MT Right – MT Left Criterion to answer “Left” LIP represents ∫ dt of momentary motion evidence • • Momentary evidence is a spike rate difference from area MT • The accumulated evidence used by the monkey is in area LIP • How and where is the integral computed? • How is the bound set? • How is a bound crossing detected?

Probabilistic categorization task: 4-card stud Delay & Sacade 4th Stim 2000 ms 3rd Stim 1500 ms 2nd Stim 1st Stim & 1000 ms Targets Time 500 ms Fixation 0 ms Tianming Yang

Favoring Favoring Green Red - �� -0.9 -0.7 -0.5 -0.3 0.3 0.5 0.7 0.9 + � Weight of evidence in favor of red (log 10 likelihood ratio) 0.61 0.39 0.9 -0.9 -0.2 -0.9 0.7

From sensorimotor integration to cognition and its disorders Potential Sensory Motor behavior evidence output or plan Prior knowledge Expected payoff Urgency

From sensorimotor integration to cognition and its disorders Sensory Motor Area LIP evidence response

From sensorimotor integration to cognition and its disorders Sensory Oculomotor Motor Area MT Area LIP evidence System output

From sensorimotor integration to cognition and its disorders Sensory Motor evidence response � dt Evanescent Plans for sensory stream the future Leaky integration  confusion

Turing’s strategy: sequential analysis ( ) � � � 1 � = + 3.0 db � 13 � Hypothesis : Messages encrypted 10 log 10 match ( ) � � � 1 � � Weight of evidence WOE = 26 � ( ) in favor of � � by Enigma devices in same state � 12 = � 0.17 db common rotor setting 13 � � 10 log 10 non - match � ( ) � � 25 � � � 26 of common settings (decibans) K C Y W D K D O P E D B A I Q S D F M K C N F A E O I E N C V N S D F N Weight of evidence in favor E N C H P D N C O E N A S H Q E N D N C K R N D N Q I O M Z F J K C P Q 5 0

Turing’s strategy: sequential analysis ( ) � � � 1 � = + 3.0 db � 13 � Hypothesis : Messages encrypted 10 log 10 match ( ) � � � 1 � � Weight of evidence WOE = 26 � ( ) in favor of � � by Enigma devices in same state � 12 = � 0.17 db common rotor setting 13 � � 10 log 10 non - match � ( ) � � 25 � � � 26 of common settings (decibans) K C Y W D K D O P E D B A I Q S D F M K C N F A E O I E N C V N S D F N Weight of evidence in favor E N C H P D N C O E N A S H Q E N D N C K R N D N Q I O M Z F J K C P Q 5 0

Turing’s strategy: sequential analysis ( ) � � � 1 � = + 3.0 db � 13 � 10 log 10 match ( ) � � � 1 � � Weight of evidence WOE = 26 � ( ) in favor of � � � 12 = � 0.17 db common rotor setting 13 � � 10 log 10 non - match � ( ) � � 25 � � � 26 of common settings (decibans) K C Y W D K D O P E D B A I Q S D F M K C N F A E O I E N C V N S D F N K C Y W D K D O P E D B A I Q S D F M K C N F A E O I E N C V N S D F N Weight of evidence in favor E N C H P D N C O E N A S H Q E N D N C K R N D N Q I O M Z F J K C P Q E N C H P D N C O E N A S H Q E N D N C K R N D N Q I O M Z F J K C P Q 5 0

Variable response to weak RIGHTWARD motion LEFT preferring MT neurons 0.08 RIGHT preferring 0.07 MT neurons 0.06 Frequency of 0.05 Probability occurrence 0.04 0.03 0.02 0.01 0 0 10 20 30 40 50 Response (spikes/sec) Response (spikes/s)

Difference in spike rate is proportional to the logarithm of the likelihood ratio Distribution of response 0.06 DIFFERENCES, right-left , for 0.05 rightward motion Frequency of 0.04 occurrence 0.03 0.02 0.01 0 -40 -20 0 20 40 Response difference (spikes/s)

Amount of accumulated evidence required to Accumulated choose “RIGHT” difference (R-L) Log of Likelihood Ratio Weight of evidence 0 Decibans Belief Amount of accumulated 0 0.2 0.4 0.6 0.8 evidence required to Time (s) choose “LEFT”

Random walk to bounds at ±A n � Y n = random walk or diffusion X i i = 1 � � � � M X ( � ) � E e � X � = f ( x ) e � x dx def. of MGF for X � �� n ( � ) MGF for sums M Y n ( � ) = M X � Y � stopped accumulation + e � A + (1 � P + ) e � � A MFG for � Y ( � ) = P M � Y

Stochastic processes: partial sums and Wald’s martingale

Stochastic processes: partial sums and Wald’s martingale

Wald’s martingale & identity � � ( n + 1) ( � ) e � Y n + 1 Y 1 , Y 2 , … , Y n � � � � = E M X E Z n + 1 Y 1 , Y 2 , … , Y n � � � � � ( n + 1) ( � ) e � ( Y n + X n + 1 ) � = E M X � � � n ( � ) e � Y n e � X n + 1 ] = E [ M X � 1 ( � ) M X � 1 ( � ) e � X n + 1 ] ��� = E [ Z n M X = M X � 1 ( � ) Z n E e � X n + 1 � � � � = Z n [ ] = E M � � � n ( � ) e � Y n E Z n � ������� � X � n ( � ) E e � Y n � � = M � � X = M � n ( � ) M Y n ( � ) X = 1