Announcements HW 3 is out, due Nov 5th Project instruction is out - PowerPoint PPT Presentation

Announcements Ø HW 3 is out, due Nov 5’th Ø Project instruction is out • Format: proposal (5’) + presentation (10’) + report (25’) • Proposal due Nov 7’th -- mainly to check you formed a team and have some ideas about what to do • We have some suggested topics, but you are more encouraged to find your own 1

CS6501: T opics in Learning and Game Theory (Fall 2019) Scoring Rules Instructor: Haifeng Xu

Outline Ø Recap: Prediction Markets Ø Scoring Rule and its Characterization Ø Connection to Prediction Markets 3

Prediction Markets A prediction market is a financial market that is designed for event prediction via information aggregation Ø Payoffs of the traded contract are determined by outcomes of future events . . . $1 iff 𝑓 # $1 iff 𝑓 " contracts We design a market maker by specifying the payment for bundles of contracts. 4

Example: Logarithmic Market Scoring Rule (LMSR [Hanson 03, 06]) . . . $1 iff 𝑓 # $1 iff 𝑓 " Ø Define value function ( 𝑟 = (𝑟 " , ⋯ , 𝑟 # ) is current sales quantity) 𝑊 𝑟 = 𝑐 log ∑ 0∈[#] 𝑓 4 5 /7 Parameter 𝑐 adjusts liquidity Ø Price function 𝑓 4 : /7 ∑ 0∈[#] 𝑓 4 5 /7 = 𝜖𝑊(𝑟) 𝑞 9 𝑟 = 𝜖𝑟 9 Ø To buy 𝑦 ∈ ℝ # amount, a buyer pays: 𝑊 𝑟 + 𝑦 − 𝑊(𝑟) • Negative 𝑦 9 ’s mean selling contracts to MM • Negative payment means market maker pays the buyer • Market starts with 𝑊 0 = 𝑐 log 𝑜 5

Properties of LMSR Fact. The optimal amount an expert purchases is the amount that moves the market price to her belief 𝜇 . Her expected utility of purchasing this amount is always non-negative. C D(4EF ∗ ) Ø I.e., should purchase 𝑦 ∗ such that = 𝜇 9 ∗ C F : Ø Market efficiency Fact. Worst case market maker loses is 𝑐 log 𝑜 (i.e., bounded). 6

Price Curve as a Function of Share Quantities 7

Examples of LMSR in Practice Ø Has been implemented by several prediction markets • E.g., InklingMarkets, Washington Stock Exchange, BizPredict, Net Exchange, and (reportedly) at YooNew. 8

Markets can potentially be a very effective forecasting tool Big on-going project: “replication market” for DARPA SCORE project 9

Connection between LMSR and Exponential Weight Updates (EWU) 10

Recap: Exponential Weight Update Ø Played for 𝑈 rounds; each round selects an action 𝑗 ∈ [𝑜] Ø Maintains weights over 𝑜 actions: 𝑥 K 1 , ⋯ , 𝑥 K (𝑜) Ø Observe cost vector 𝑑 K , and update 𝑥 KE" 𝑗 = 𝑥 K 𝑗 ⋅ 𝑓 OPQ R 9 , ∀𝑗 ∈ [𝑜] . . . Action 1, 𝑥 K (1) Action 𝑜 , 𝑥 K (𝑜) Action 2, 𝑥 K (2) 11

Recap: Exponential Weight Update Ø Played for 𝑈 rounds; each round selects an action 𝑗 ∈ [𝑜] Ø Maintains weights over 𝑜 actions: 𝑥 K 1 , ⋯ , 𝑥 K (𝑜) Ø Observe cost vector 𝑑 K , and update 𝑥 KE" 𝑗 = 𝑥 K 𝑗 ⋅ 𝑓 OPQ R 9 , ∀𝑗 ∈ [𝑜] . . . Action 1, 𝑥 K (1) Action 𝑜 , 𝑥 K (𝑜) Action 2, 𝑥 K (2) 𝑥 KE" 𝑗 = 𝑥 K 𝑗 ⋅ 𝑓 OPQ R 9 = [𝑥 KO" 𝑗 ⋅ 𝑓 OPQ RUV 9 ] ⋅ 𝑓 OPQ R 9 = ⋯ = 𝑓 OPW R 9 where 𝐷 K 𝑗 = ∑ YZK 𝑑 Y (𝑗) 12

Recap: Exponential Weight Update Ø Played for 𝑈 rounds; each round selects an action 𝑗 ∈ [𝑜] Ø Maintains weights over 𝑜 actions: 𝑥 K 1 , ⋯ , 𝑥 K (𝑜) Ø Observe cost vector 𝑑 K , and update 𝑥 KE" 𝑗 = 𝑥 K 𝑗 ⋅ 𝑓 OPQ R 9 , ∀𝑗 ∈ [𝑜] Ø At round 𝑢 + 1 , select action 𝑗 with probability 𝑓 OPW R 9 𝑥 K (𝑗) = 𝑋 ∑ 0∈[#] 𝑓 OPW R 0 K where 𝐷 K = ∑ YZK 𝑑 K is the accumulated cost vector 13

Recap: Exponential Weight Update Ø Played for 𝑈 rounds; each round selects an action 𝑗 ∈ [𝑜] Ø Maintains weights over 𝑜 actions: 𝑥 K 1 , ⋯ , 𝑥 K (𝑜) Ø Observe cost vector 𝑑 K , and update 𝑥 KE" 𝑗 = 𝑥 K 𝑗 ⋅ 𝑓 OPQ R 9 , ∀𝑗 ∈ [𝑜] Ø At round 𝑢 + 1 , select action 𝑗 with probability 𝑓 OPW R 9 𝑥 K (𝑗) = 𝑋 ∑ 0∈[#] 𝑓 OPW R 0 K where 𝐷 K = ∑ YZK 𝑑 K is the accumulated cost vector This looks very much like the price function in LMSR ( 𝑟 is the accumulated sales quantity) 𝑓 4 : /7 𝑞 9 = ∑ 0∈[#] 𝑓 4 5 /7 14

EWU vs LMSR Ø Exponential Weight Update Ø LMSR • 𝑜 actions • 𝑜 contracts (i.e., outcomes) • Maintain weight 𝑥 K (𝑗) • Maintain prices 𝑞(𝑗) • Total cost 𝐷 ` 𝑗 = ∑ KZ` 𝑑 K (𝑗) • Total shares sold 𝑟 𝑗 • Select 𝑗 with prob • Price of contract 𝑗 𝑓 4 : /7 𝑓 OPW R 9 𝑞 9 = 𝑞 9 = ∑ 0∈[#] 𝑓 4 5 /7 ∑ 0∈[#] 𝑓 OPW R 0 • Weights reflect how good an • Prices reflect how probable is an action is event • Care about worst case regret • Care about worst case MM loss 𝐷 ` Alg − min 𝐷 ` (𝑗) ($ received) − max 𝑟(𝑗) 9 9 15

Ø LMSR is just one particular automatic MM • Similar relation holds for other market markers and no-regret learning algorithms (see [Chen and Vaughan 2010]) Ø Next: will study other “good” scoring rules, and see why they work 16

Outline Ø Recap: Prediction Markets Ø Scoring Rule and its Characterization Ø Connection to Prediction Markets 17

Consider a Simpler Setting Ø We (designer) want to learn the distribution of random var 𝐹 ∈ [𝑜] • 𝐹 will be sampled in the future Ø We have no samples from 𝐹 ; Instead, we have an expert/predictor who has a predicted distribution 𝜇 ∈ Δ # Ø We want to incentivize the expert to truthfully report 𝜇 𝜇 18

Consider a Simpler Setting Ø We (designer) want to learn the distribution of random var 𝐹 ∈ [𝑜] • 𝐹 will be sampled in the future Ø We have no samples from 𝐹 ; Instead, we have an expert/predictor who has a predicted distribution 𝜇 ∈ Δ # Ø We want to incentivize the expert to truthfully report 𝜇 Example Ø 𝐹 is whether UVA will win NCAA title in 2020 Ø Expert is the UVA coach Ø Expert’s prediction does not need to be perfect • But, better than the designer who knows nothing Ø Assume expert will not give you truthful info for free 19

Idea: “Score” Expert’s Report Will reward the expert certain amount 𝑇(𝑗; 𝑞) where: (1) 𝑞 is the expert’s report (does not have to equal 𝜇 ); (2) 𝑗 ∈ [𝑜] is the event realization Not like a prediction market yet, but will see later they are related 20

Idea: “Score” Expert’s Report Will reward the expert certain amount 𝑇(𝑗; 𝑞) where: (1) 𝑞 is the expert’s report (does not have to equal 𝜇 ); (2) 𝑗 ∈ [𝑜] is the event realization Q : what is the expert’s expected utility? Ø Expert believes 𝑗 ∼ 𝜇 Ø Expected utility 𝔽 9∼j 𝑇 𝑗; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ 𝑇(𝑗; 𝑞) = 𝑇(𝜇; 𝑞) 21

Idea: “Score” Expert’s Report Will reward the expert certain amount 𝑇(𝑗; 𝑞) where: (1) 𝑞 is the expert’s report (does not have to equal 𝜇 ); (2) 𝑗 ∈ [𝑜] is the event realization Q : what is the expert’s expected utility? Ø Expert believes 𝑗 ∼ 𝜇 Ø Expected utility 𝔽 9∼j 𝑇 𝑗; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ 𝑇(𝑗; 𝑞) = 𝑇(𝜇; 𝑞) Q : what 𝑇(𝑗; 𝑞) function can elicit truthful report 𝜇 ? l∈m n [∑ 9∈[#] 𝜇 9 ⋅ 𝑇(𝑗; 𝑞)] Ø When expert finds that 𝜇 = arg max Ø Ideally, 𝜇 is the unique maximizer 22

Proper Scoring Rules Definition. The “scoring rule” 𝑇(𝑗; 𝑞) is [strictly] proper if truthful report 𝑞 = 𝜇 [uniquely] maximizes expected utility 𝑇(𝜇; 𝑞) . Ø Expert is incentivized to report truthfully iff 𝑇(𝑓; 𝑞) is proper Observations. 𝑇 𝑗; 𝑞 = 0 is a trivial proper scoring fnc 1. 2. Proper scores are closed under affine transformation I.e., if 𝑇 𝑗; 𝑞 is [strictly] proper, so is 𝛽 ⋅ 𝑇 𝑗; 𝑞 + 𝛾 • for any constant 𝛽 ≠ 0, 𝛾 Ø Thus, typically, strict properness is desired 23

Examples of Scoring Rules Example 1 [Log Scoring Rule] Ø 𝑇 𝑗; 𝑞 = log 𝑞 9 Ø 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 Ø Negative, but okay – can always add a constant Ø Properness requires 𝜇 = arg max l∈m n 𝑇(𝜇; 𝑞) 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 = ∑ 9∈[#] 𝜇 9 log 𝑞 9 − log 𝜇 9 + ∑ 9∈[#] 𝜇 9 log 𝜇 9 24

Examples of Scoring Rules Example 1 [Log Scoring Rule] Ø 𝑇 𝑗; 𝑞 = log 𝑞 9 Ø 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 Ø Negative, but okay – can always add a constant Ø Properness requires 𝜇 = arg max l∈m n 𝑇(𝜇; 𝑞) 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 = ∑ 9∈[#] 𝜇 9 log 𝑞 9 − log 𝜇 9 + ∑ 9∈[#] 𝜇 9 log 𝜇 9 j : = − ∑ 9∈[#] 𝜇 9 ⋅ log l : − 𝐹𝑜𝑢𝑠𝑝𝑞(𝜇) 25

Examples of Scoring Rules Example 1 [Log Scoring Rule] Ø 𝑇 𝑗; 𝑞 = log 𝑞 9 Ø 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 Ø Negative, but okay – can always add a constant Ø Properness requires 𝜇 = arg max l∈m n 𝑇(𝜇; 𝑞) 𝑇 𝜇; 𝑞 = ∑ 9∈[#] 𝜇 9 ⋅ log 𝑞 9 = ∑ 9∈[#] 𝜇 9 log 𝑞 9 − log 𝜇 9 + ∑ 9∈[#] 𝜇 9 log 𝜇 9 j : = − ∑ 9∈[#] 𝜇 9 ⋅ log l : − 𝐹𝑜𝑢𝑠𝑝𝑞(𝜇) KL-divergence 𝐿𝑀(𝜇; 𝑞) (a.k.a. relative entropy) • Measures the distance between two distributions Always non-negative, and equals 0 only when 𝑞 = 𝜇 • 26