L ECTURE 28: G AME T HEORY 3 I NSTRUCTOR : G IANNI A. D I C ARO M - PowerPoint PPT Presentation

15-382 C OLLECTIVE I NTELLIGENCE – S18 L ECTURE 28: G AME T HEORY 3 I NSTRUCTOR : G IANNI A. D I C ARO

M IXED N ASH E QUILIBRIUM R P S 1 3 ,1 3 , 1 3 , 1 3 ,1 3 , 1 3 R 0,0 -1,1 1,-1 Finding ME: & ,𝜌 ' & ,𝜌 ( & , be the probabilities of the pure Let 𝜌 % P 1,-1 0,0 -1,1 § & + 𝜌 ' & + 𝜌 ( & = 1 strategy mix for player 𝑗 = 1,2 , 𝜌 % -1,1 1,-1 0,0 § A mixed strategy equilibrium needs to make player S 𝑗 indifferent among all three of his strategies (i.e. same expected utility) R P S à Find player 𝑗 expected utilities as a function of § the parameters of the mixed strategy and set the R 0,0 -2,2 1,-1 parameters in order to satisfy the previous requirement 2,-2 0,0 -1,1 P § In symmetric zero-sum games the expected utility of the players at equilibrium is zero à This property can be used to rule out equilibrium candidates (just -1,1 1,-1 0,0 S 15781 Fall 2016: Lecture 22 check if one player has a positive utility!) 2

G AME OF CHICKEN http://youtu.be/u7hZ9jKrwvo § Each player, in attempting to secure his best outcome, risks the worst § Every player wants to dare , but only if the other chickens out ! § A mediator would help… 3

G AME OF CHICKEN § Social welfare is the sum of utilities Dare Chicken § Optimal social welfare = 6 0,0 4,1 Dare § Pure NE: (C,D) and (D,C), social welfare = 5 1,4 3,3 Chicken § Mixed NE: both ( . / , . / ), social welfare = 4 § Can we do better? Players are independent so far … 4

C ORRELATED EQUILIBRIUM § A “trusted” authority / mediator chooses a pair of strategies (𝑡 1 , 𝑡 2 ) according to a probability distribution 𝑞 over 𝑇 2 (it can be generalized to 𝑜 players) Robert Aumann Nobel prize, 2005 § The mediator “flips a coin” / draw according to the distribution 𝑞( 𝑡 . , 𝑡 / ) and, based on the outcome, tells the players which pure strategy to use based 5

C ORRELATED EQUILIBRIUM § à The trusted party only tells each player what to do, but it does not reveal what the other party is supposed to do! § The distribution 𝒒 is known to the players : each player knows the probability of observing a strategy profile and assumes the other player will follow mediator’s instructions § à Posterior conditional probability is known: Pr [𝑡 & |𝑡 ; ] § It is a Correlated Equilibrium (CE) if no player wants to deviate from the trusted party’s instructions, such that choices are correlated § à Find distribution 𝑞 that guarantees a CE 6

C ORRELATED EQUILIBRIUM § Common knowledge: Distribution 𝑞 (is CE) o (D,D): 0 Dare Chicken o (D,C): . > 0,0 7,2 Dare o (C,D): . > o (C,C): . > 2,7 6,6 Chicken § If Player 2 is told to play D, then P2 knows that the outcome must be (C,D) and that Player 1 will obey the instructions à P1 plays C ü Based on this, Player 2 has no incentive to change from playing D, as given 7

C ORRELATED EQUILIBRIUM Chicken Dare § Distribution 𝑞 (is CE) o (D,D): 0 Dare 0,0 7,2 o (D,C): . > o (C,D): . > Chicken o (C,C): . 2,7 6,6 > § If Player 2 is told to play C, then 2 knows that the outcome must be (D,C) or (C,C) with equal probability. § Player’s 2 expected utility on playing C conditioned on the fact that he is told to play C (and Player 1 will obey instructions) is: . . . . / 𝑣 / 𝐸, 𝐷 + / 𝑣 / 𝐷, 𝐷 = / 2 + / 6 = 4 § If Player 2 deviates from instructions and plays D: 𝑣 / = 3.5 < 4 ü It’s better to follow the instructions! 8

C ORRELATED EQUILIBRIUM Chicken Dare § Distribution 𝑞 (is CE) o (D,D): 0 Dare o (D,C): . 0,0 7,2 > o (C,D): . > o (C,C): . Chicken 2,7 6,6 > § Player 2 does not have incentive to deviate § Since the game is symmetric , also Player 1 does not have incentive to deviate § → Correlated equilibrium! § Expected reward per player: (1/3)*7 + (1/3)*2 + (1/3)*6 = 5 § Mixed strategy NE: 4*(2/3), which is < 5 § Social welfare: 30/3 9

C ORRELATED EQUILIBRIUM § Let 𝑂 = {1,2} for simplicity § A mediator chooses a pair of strategies (𝑡 . ,𝑡 / ) according to a distribution 𝑞 over 𝑇 / § Reveals 𝑡 . to player 1 and 𝑡 / to player 2 § When player 1 gets 𝑡 . ∈ 𝑇 , he knows that the distribution over strategies of 2 is Pr 𝑡 / 𝑡 . = Pr 𝑡 . ∧ 𝑡 / 𝑞 𝑡 . , 𝑡 / = M ) ∑ Pr 𝑡 . 𝑞(𝑡 . , 𝑡 / P ∈( N O 10

C OMPUTING CE S TRATEGY § Player’s 1 strategy 𝑡 . is a best response if its expected utility cannot be unilaterally improved based on what he knows: M ∈ 𝑇 M ,𝑡 / ) Q Pr 𝑡 / 𝑡 . 𝑣 . 𝑡 . ,𝑡 / ≥ Q Pr 𝑡 / 𝑡 . 𝑣 . (𝑡 . , ∀𝑡 . N O ∈( N O ∈( § Equivalently, replacing using Bayes’ rule M ,𝑡 / ) Q 𝑞 𝑡 . , 𝑡 / 𝑣 . 𝑡 . ,𝑡 / ≥ Q 𝑞 𝑡 . ,𝑡 / 𝑣 . (𝑡 . N O ∈( N O ∈( § 𝑞 is a correlated equilibrium (CE) if both players are best responding 11

CE A S LP § Can compute CE via linear programming in polynomial time! find 𝑞 𝑡 . , 𝑡 / M , 𝑡 / ∈ 𝑇, s.t. ∀𝑡 . , 𝑡 . M ,𝑡 / ) Q 𝑞 𝑡 . ,𝑡 / 𝑣 . 𝑡 . ,𝑡 / ≥ Q 𝑞 𝑡 . ,𝑡 / 𝑣 . (𝑡 . N O ∈U N O ∈U M ∈ 𝑇, ∀𝑡 . , 𝑡 / , 𝑡 / M ) Q 𝑞 𝑡 . ,𝑡 / 𝑣 / 𝑡 . ,𝑡 / ≥ Q 𝑞 𝑡 . ,𝑡 / 𝑣 / (𝑡 . ,𝑡 / N T ∈U N T ∈U Q 𝑞 𝑡 . , 𝑡 / = 1 N T ,N O ∈( ∀𝑡 . , 𝑡 / ∈ 𝑇, 𝑞 𝑡 . , 𝑡 / ∈ [0,1] 12

B EST W ELFARE CE § Adding an objective (linear) function f , the best correlated equilibrium (e.g., max welfare) can be found max 𝑔(𝑞 𝑡 . , 𝑡 / ;𝑣 . , 𝑣 / ) s.t. ∀𝑡 . , 𝑡 . M , 𝑡 / ∈ 𝑇, M ,𝑡 / ) Q 𝑞 𝑡 . ,𝑡 / 𝑣 . 𝑡 . ,𝑡 / ≥ Q 𝑞 𝑡 . ,𝑡 / 𝑣 . (𝑡 . N O ∈U N O ∈U M ∈ 𝑇, ∀𝑡 . , 𝑡 / , 𝑡 / M ) Q 𝑞 𝑡 . ,𝑡 / 𝑣 / 𝑡 . ,𝑡 / ≥ Q 𝑞 𝑡 . ,𝑡 / 𝑣 / (𝑡 . ,𝑡 / N T ∈U N T ∈U Q 𝑞 𝑡 . , 𝑡 / = 1 N T ,N O ∈( ∀𝑡 . , 𝑡 / ∈ 𝑇, 𝑞 𝑡 . , 𝑡 / ∈ [0,1] 13

I MPLEMENTATION OF CE § Instead of a mediator, use a hat! § Balls in hat are labeled with “chicken” or “dare”, each blindfolded player takes a ball Which balls implement D the distribution 𝑞 before ? D C C C D C C 1. 1 chicken, 1 dare 2. 2 chicken, 1 dare 3. 2 chicken, 2 dare 4. 3 chicken, 2 dare E.g., An automatic trusting authority can be implemented using cryptographic algorithms 14

CE VS . NE What is the relation between CE and NE? CE ⇒ NE 1. NE ⇒ CE 2. NE ⇔ CE 3. NE ∥ CE 4. § For any pure strategy NE, there is a corresponding correlated equilibrium yielding the same outcome. § For any mixed strategy NE, there is a corresponding correlated equilibrium yielding the same distribution of outcomes. § From Nash theorem, “all” games have a mixed strategies NE”. Since a NE implies a CE, a CE always exist 15

A DIFFERENT TYPE OF GAMES : S TACKELBERG GAMES L R § Playing up is a dominant strategy for row player U 1,1 3,0 § Row player plays up § Column player would then play left 0,0 2,1 § Therefore, (1,1) is the only Nash D equilibrium outcome 16

C OMMITMENT IS GOOD § Suppose the game is played L R sequentially as follows: o Row player commits to playing U 1,1 3,0 a row o Column player observes the commitment and chooses a D 0,0 2,1 column § Row player can commit to playing Down: Column player will play Right and the Row player gets now a better reward! 17

C OMMITMENT TO MIXED STRATEGY § By committing to a mixed strategy , 𝜌 _ = 0 𝜌 % = 1 row player can get even better and guarantee a reward of almost 2.5: 𝜌 ] = 0.49 1,1 3,0 0.49×3 + 0.51×2 § Stackelberg strategy (1934) 𝜌 ^ = 0.51 0,0 2,1 § Rooted in duopoly scenarios § Player 1 ( Leader ) moves at the start of the game. Then use backward induction to find the subgame perfect equilibrium. § First for any output of leader, find the strategy of Follower that maximizes its payoff (its expected best reply). § Next, find the strategy of leader that maximizes leader player utility, given the strategy of follower 18

C OMPUTING S TACKELBERG § Theorem [Conitzer and Sandholm, 2006] : In 2-player normal form games, an optimal Stackelberg strategy can be found in polynomial time § Theorem: The problem is NP-hard when the number of players is ≥ 3 19

T RACTABILITY : 2 PLAYERS § For each pure strategy 𝑡 / of the follower, we compute via the LP below a mixed strategy 𝑦 . for the leader such that: o Playing 𝑡 / is a best response for the follower o Under this constraint, 𝑦 . is optimal (for follower) ∗ that maximizes leader’s utility § Choose 𝑦 . max ∑ 𝑦 . 𝑡 . 𝑣 . (𝑡 . ,𝑡 / ) N T ∈( s.t. M ∈ 𝑇, M ∑ . , 𝑡 / ≥ ∑ 𝑦 . 𝑡 . 𝑣 / 𝑡 𝑦 . 𝑡 . 𝑣 / 𝑡 . , 𝑡 / ∀𝑡 / N T ∈( N T ∈( ∑ 𝑦 . 𝑡 . = 1 N T ∈( ∀𝑡 . ∈ 𝑇, 𝑦 . 𝑡 . ∈ [0,1] 20

A PPLICATION : SECURITY § Attacker monitors defender and tries to maximize damage , while a defender deploys resources to minimize damage based on knowledge of what the attacker would like to obtain § Airport security: deployed at LAX § Federal Air Marshals § Coast Guard § Idea: o Defender commits to mixed strategy o Attacker observes and best responds 21