Human Learning in Dynamic Human Learning in Dynamic Environments - PowerPoint PPT Presentation

Human Learning in Dynamic Human Learning in Dynamic Environments Cleotilde (Coty) Gonzalez Dynamic Decision Making Laboratory www.cmu.edu/DDMLab d /DDML b Social and Decision Sciences Department Carnegie Mellon University Research supported by the National Science Foundation : Human and Social Dynamics: Decision, Risk, and Uncertainty

Dynamic Environments • Combat missions, Production scheduling, Fire fighting, Emergency dispatch, Air-traffic control • Complex Number of components: alternatives, events, courses of o action, outcomes Uncertainty: All possible states of the world and o outcomes are unavailable, incomplete, and difficult to imagine g Constraints: limited time, knowledge, resources, human o capacity • • Dynamic Complexity Dynamic Complexity Arises from the interactions of components over time o Environment is autonomous. All is change at many g y o different time scales Learning from our actions: feedback delays o

Dynamic Decision Making: A Closed-Loop view delay Hypothesize Symptoms illnesses and run tests delay delay External event Test Health Health results results delay delay Diagnosis Diagnosis Treatment Treatment delay

Learning in dynamic systems is hard • People remain suboptimal in these systems even with repeated trials, unlimited time and performance incentives (Sterman,1994; Diehl & Sterman 1995) Sterman, 1995). • We have difficulty processing feedback. Feedback delay is a problem for learning F db k d l bl f l (Brehmer, 1992; Sterman, 1989).

But… how do we learn in dynamic environments? environments? • Decision Makers recognize typical situations and typical responses. Decision makers use their past knowledge D i i k th i t k l d and adapt their strategies “on the fly”. Chess studies, Expertise: Chase & Simon, 1973 Chess studies Expertise: Chase & Simon 1973 o Adaptive Decision Making: Payne, Bettman, & Johnson, 1993 o Decision making under uncertainty: “Case-Based Decision Decision making under uncertainty Case Based Decision o o Theory” , Gilboa and Schmeidler, 1995 Theory of automaticity: Logan, 1988 o “Recognition-Primed Decision Making” (RPDM): Intuition, Mental o simulations, Klein et al., 1993; Klein, 1998

Pattern recognition is easier if you have experience i

Instance Based Learning Theory (Gonzalez, Lerch, & Lebiere, 2003) • RECOGNITION OF FAMILIAR PATTERNS Determining the similarity between a situation and past o experience Identifying ‘typical’ situations and responses y g yp p o • ACQUIRING CAUSE-EFFECT KNOWLEDGE Q Accumulation of instances with practice in a task o Improvement of decision making by bootstrapping on previous o k knowledge l d Implemented in ACT-R (Anderson and Lebiere, 1988)

IBLT: WHAT do we learn? Situation Decision Outcome Action- Situation- Outcome Decision Cycle Cycle Cycle Cycle Future Decisions S D O Blending Similarity S D O of past Outcomes Outcomes S S D D O O S D O Time F Feedback db k Environment

IBLT: HOW do we learn?

ACT-R (Anderson & Lebiere, 1998) (A d & L bi 1998) The 2x2 levels of ACT-R h l l f Declarative Memory Procedural Memory Chunks: declarative Productions: If facts facts (cond) Then (action) (cond) Then (action) Symbolic A ti Activation of chunks ti f h k Conflict Resolution (likelihood of (likelihood of use) retrieval) S bS SubSymbolic b li

IBLT models compare to human decision making: making: • In dynamic resource allocation tasks (Gonzalez et al., 2003) • In supply chain management control (Martin, Gonzalez & Lebiere 2004) Gonzalez & Lebiere, 2004) • In repeated choice tasks (Lebiere, Gonzalez & Martin, 2007) 2007) • But there is long way to go to demonstrate: generalizability and utility of IBLT g y y

Decision Making Games (DMGames) used for experimentation for experimentation • DMGames embody the essential characteristics of • DMGames embody the essential characteristics of real-world decision environments o Interactive o Interactive o Repeated and interrelated decisions o External events and team interactions E t l t d t i t ti • Help compress time and space – speed up learning • Help manipulate experience - learn from simulated cases and on-demand repeated practice • No risk to individuals and they are FUN. k d d l d h

DMGames used in behavioral research in the DDMlab Military Command and Control Military Command and Control Real-time resource allocation Real-time resource allocation Real time resource allocation Real time resource allocation Medical Medical ed ca ed ca Diagnosis Diagnosis Supply- Supply- Chain Chain Chain Chain Fire Fire Management Management Fighting Fighting

MEDIC: Learning tools that represent the dynamics of medical diagnosis (Gonzalez & Vrbin, 2007) y f g ( , ) • Concepts adapted from Kleinmuntz (1985): Task complexity (numerous diseases and symptoms) Task complexity (numerous diseases and symptoms) o Disease base rates o Time pressure o Test diagnosticity o Treatment effectiveness o Treatment risk Treatment risk o o • Additions: Feedback delays (e.g. receiving test results) o • With the potential for: Dynamic diagnostic cues o Dynamic symptoms o

MEDIC demo

Factors that influence Learning in dynamic systems y • Time constraints (Gonzalez, 2004) • Workload (Gonzalez, 2005) • The similarity and diversity of experiences (Gonzalez and y y p Quesada, 2004; Gonzalez and Madhavan, in preparation) • Our inherent cognitive abilities (Gonzalez, Thomas and Vanyukov 2004) Vanyukov, 2004) • The type of feedback (Gonzalez, 2005) • Our difficulty in understanding simple stock and flow Our difficulty in understanding simple stock and flow structures (Cronin and Gonzalez, 2005; Cronin, Gonzalez and Sterman, 2006; Gonzalez, Sterman and Cronin, in preparation)

Experiment 1: probabilities • MEDIC incorporated: o Symptoms-disease associations from 0.1 to 0.9 o Delay in test results y o Time pressure due to patient’s declining health in real-time o Deterministic treatment needed to be provided • N=12, students, paid flat rate N , students, pa d flat rate • Each student resolved 56 cases

Results

Treatment

Results- test diagnosticity

Disease base rates

Diagnosticity per disease

Experiment 1: Conclusions • Students did learn – not perfectly • Showed knowledge of probabilities, tested for the more diagnostic cues, and diagnosed very closely to the real state of the diseases. f . • What is the role of feedback and how would that interact with the symptom-probability matrix?

Experiment 2: Probabilities and f feedback db k • MEDIC: • Symptomology table: Probability or Certainty • either detailed feedback or no feedback • Participants were assigned to one of four conditions: probabilities, full feedback (P1) -26 o certainty full feedback (P2)-30 certainty, full feedback (P2) 30 o o certainty, no feedback (P3)-25 o probabilities, no feedback (P4)- 29 o • N= 110 Participants were paid a flat dollar amount

P Probability b bili Certainty C t i t Disease 1 Disease 2 Disease 3 Disease 4 0.25 0.25 0.25 0.25 Base Rates 0.0 0.0 0.0 0.0 Symptom 1 1.0 0.0 0.0 0.0 Symptom 2 1.0 1.0 0.0 0.0 Symptom 3 0.0 0.0 1.0 0.0 Symptom 4

Test diagnosticity - probability condition

Test diagnosticity – Certainty condition

Diagnosticity per disease

Experiment 2: Conclusions • Full feedback was helpful in the probabilistic environment and did not make a difference in the i t d did t k diff i th certain environment • We now know that: with repeated trials, students p , learn in probabilistic environments with time constraints and feedback delays • Feedback helps in probabilistic environments Feedback helps in probabilistic environments • Probabilistic environments are not the main reason for poor learning in dynamic tasks

Basic Building Blocks of Dynamic Decision Making Tasks Making Tasks • Stocks (accumulations) • Flows that increase (Inflow) or decrease (Outflow) the stock • Feedback Delays & multiple relationships • Environmental or external effects • Multiple decisions about flows These problems of dynamic control over time are important to human life: keeping a healthy weight, bank p p g y g accounts, company inventory, stress levels, climate change etc.

Humans suffer of poor understanding of accumulation: Stock-Flow failure accumulation: Stock Flow failure Cronin, Gonzalez & Sterman, 2008 ; Cronin & Gonzalez, 2007; Cronin, Gonzalez and Sterman, 2006; Sweeney & Sterman, 2000 St 2000; Sterman, 2002; 2002

Weight as balance between consumed and expended energy expended energy 1. When eaten most? 2. When exercised most? 3. When weight highest? 4. When weight lowest? 4 g

Blood glucose level as balance between glucagon and insulin production glucagon and insulin production 1. When most glucagon? g g 2. When most insulin? 3. When glucose level 3. When glucose level highest? 4. When glucose level 4. When glucose level lowest?