Inspections for Decision Makers (or: you may fool me, but not hurt - PowerPoint PPT Presentation

Inspections for Decision Makers (or: you may fool me, but not hurt me) Federico Echenique and Eran Shmaya California Institute of Technology November 10, 2007 SISHOO Nov. 10th Inspections for decision makers

Existing result: Inspections are manipulable ◮ Outcome ω ∈ Ω governed by a prob. distribution µ . ( ω = ( z 0 , z 1 , . . . ), a seq. of outcomes) ◮ A putative expert claims the distribution is ν . SISHOO Nov. 10th Inspections for decision makers

Existing result: Inspections are manipulable ◮ Outcome ω ∈ Ω governed by a prob. distribution µ . ( ω = ( z 0 , z 1 , . . . ), a seq. of outcomes) ◮ A putative expert claims the distribution is ν . ◮ An inspector says she believes the expert iff ω is in test set T ν ⊆ Ω. ◮ ( T ν ) is s.t. true expert passes the test w/prob. 1. SISHOO Nov. 10th Inspections for decision makers

Existing result: Inspections are manipulable ◮ Outcome ω ∈ Ω governed by a prob. distribution µ . ( ω = ( z 0 , z 1 , . . . ), a seq. of outcomes) ◮ A putative expert claims the distribution is ν . ◮ An inspector says she believes the expert iff ω is in test set T ν ⊆ Ω. ◮ ( T ν ) is s.t. true expert passes the test w/prob. 1. Result: A false expert can always manipulate the test, and pass for all ω . (Foster & Vohra, Lehrer, Olszewski & Sandroni, Shmaya) SISHOO Nov. 10th Inspections for decision makers

We study case where inspector cares about ν because she has to make a decision. SISHOO Nov. 10th Inspections for decision makers

Inspector is a DM ◮ A putative expert claims the distribution is ν . ◮ DM cares about ω : she makes a decision a ∈ A ; her payoff depends on ( ω, a ). SISHOO Nov. 10th Inspections for decision makers

Inspector is a DM ◮ A putative expert claims the distribution is ν . ◮ DM cares about ω : she makes a decision a ∈ A ; her payoff depends on ( ω, a ). � believe ν → use a ∗ ν , optimal action for ν DM → use a ∗ reject ν π , optimal action for π. π is the DM’s existing belief about Ω SISHOO Nov. 10th Inspections for decision makers

We compare Payoff( ω, a ∗ ν ) − Payoff( ω, a ∗ π ) under two criteria: ν and π . SISHOO Nov. 10th Inspections for decision makers

Our test We show: there is a test ( T ν ) s.t. ◮ true expert passes the test w/prob. 1. SISHOO Nov. 10th Inspections for decision makers

Our test We show: there is a test ( T ν ) s.t. ◮ true expert passes the test w/prob. 1. ◮ DM can follow expert recommendation (i.e. choose a ∗ ν ). regardless of whether the expert is true. SISHOO Nov. 10th Inspections for decision makers

Our test We show: there is a test ( T ν ) s.t. ◮ true expert passes the test w/prob. 1. ◮ DM can follow expert recommendation (i.e. choose a ∗ ν ). regardless of whether the expert is true. ◮ if DM thinks expert is true, she should choose a ∗ ν SISHOO Nov. 10th Inspections for decision makers

Our test We show: there is a test ( T ν ) s.t. ◮ true expert passes the test w/prob. 1. ◮ DM can follow expert recommendation (i.e. choose a ∗ ν ). regardless of whether the expert is true. ◮ if DM thinks expert is true, she should choose a ∗ ν ◮ if DM thinks expert is false, under beliefs π , ν will not pass the test when a ∗ ν is bad for DM. (. . . if DM is patient enough) SISHOO Nov. 10th Inspections for decision makers

Our test We show: there is a test ( T ν ) s.t. ◮ true expert passes the test w/prob. 1. ◮ DM can follow expert recommendation (i.e. choose a ∗ ν ). regardless of whether the expert is true. ◮ if DM thinks expert is true, she should choose a ∗ ν ◮ if DM thinks expert is false, under beliefs π , ν will not pass the test when a ∗ ν is bad for DM. (. . . if DM is patient enough) Let me say it again. SISHOO Nov. 10th Inspections for decision makers

Model Model: ( Z , A , r , λ, π ). ◮ Z finite or countable set. ◮ Ω: infinite sequences z 1 , z 2 , . . . in Z . ◮ A : set of actions. ◮ r : Z × A → [0 , 1] : payoff function. ◮ λ ∈ (0 , 1) : discount factor. ◮ π ∈ ∆(Ω) : beliefs. SISHOO Nov. 10th Inspections for decision makers

Model A test is a function T : ∆(Ω) → subsets of Ω. A test T is type-I error free if ν ( T ( ν )) = 1 ∀ ν ∈ ∆(Ω). SISHOO Nov. 10th Inspections for decision makers

Model A strategy for DM is f : Z < N → A . f ( z 0 , . . . , z n − 1 ) is the action taken by the DM after observing ( z 0 , . . . , z n − 1 ). SISHOO Nov. 10th Inspections for decision makers

Model A strategy for DM is f : Z < N → A . f ( z 0 , . . . , z n − 1 ) is the action taken by the DM after observing ( z 0 , . . . , z n − 1 ). Payoff: � λ n r ( z n , f ( z 0 , . . . , z n − 1 )) R λ ( ω, f ) = (1 − λ ) n ∈ N from strategy f and outcome ω = ( z 0 , z 1 , . . . ). f is ν -optimal iff � f ∈ argmax R λ ( x , g ) ν (d x ) . SISHOO Nov. 10th Inspections for decision makers

Theorem There exists a type-I error free test T s.t. � lim ( R λ ( ω, g ) − R λ ( ω, f )) π ( d ω ) ≤ 0 λ → 1 T ( ν ) for every ν ∈ ∆( X ) and every ν -optimal strategy f and π -optimal strategy g. SISHOO Nov. 10th Inspections for decision makers

Merging π, ν ∈ ∆(Ω). ν merges with π if n →∞ d ( π ω | n , ν ω | n ) = 0 , lim with π -prob. 1. SISHOO Nov. 10th Inspections for decision makers

Merging π, ν ∈ ∆(Ω). ν merges with π if n →∞ d ( π ω | n , ν ω | n ) = 0 , lim with π -prob. 1. π is abs. cont. w.r.t. ν if ν ( A ) = 0 ⇒ π ( A ) = 0. Proposition (Blackwell-Dubins Theorem) If π is abs. cont. w.r.t. ν , then ν merges with π SISHOO Nov. 10th Inspections for decision makers

Our test There is T ν s.t. ν ( T ν ) = 1, and ( ν ( A ) = 0 ⇒ π ( A ) = 0) on T ν . exists by application of Lebesgue’s Decomposition Theorem. Then, on T ν , ν merges with π . If λ is large enough, payoffs under π are close. SISHOO Nov. 10th Inspections for decision makers

Our test Turns out: � π ( z 1 , . . . z n ) � T ν = ω : lim sup ν ( z 1 , . . . z n ) < ∞ n →∞ (a “likelihood ratio” test). SISHOO Nov. 10th Inspections for decision makers

Related work. ◮ Olszewski & Sandroni ◮ Al-Najjar & Weinstein ◮ Feinberg & Stewart SISHOO Nov. 10th Inspections for decision makers