CSC304 Lecture 4 Game Theory (Cost sharing & congestion games, - PowerPoint PPT Presentation

CSC304 Lecture 4 Game Theory (Cost sharing & congestion games, Potential function, Braess ’ paradox) CSC304 - Nisarg Shah 1

Recap • Nash equilibria (NE) ➢ No agent wants to change their strategy ➢ Guaranteed to exist if mixed strategies are allowed ➢ Could be multiple • Pure NE through best-response diagrams • Mixed NE through the indifference principle CSC304 - Nisarg Shah 2

Worst and Best Nash Equilibria • What can we say after we identify all Nash equilibria? ➢ Compute how “ good ” they are in the best/worst case • How do we measure “ social good ”? ➢ Game with only rewards? Higher total reward of players = more social good ➢ Game with only penalties? Lower total penalty to players = more social good ➢ Game with rewards and penalties? No clear consensus… CSC304 - Nisarg Shah 3

Price of Anarchy and Stability • Price of Anarchy (PoA) • Price of Stability (PoS) “Worst NE vs optimum” “Best NE vs optimum” Max total reward Max total reward Min total reward in any NE Max total reward in any NE or or Max total cost in any NE Min total cost in any NE Min total cost Min total cost PoA ≥ PoS ≥ 1 CSC304 - Nisarg Shah 4

Revisiting Stag-Hunt Hunter 2 Stag Hare Hunter 1 Stag (4 , 4) (0 , 2) Hare (2 , 0) (1 , 1) • Max total reward = 4 + 4 = 8 • Three equilibria ➢ (Stag, Stag) : Total reward = 8 ➢ (Hare, Hare) : Total reward = 2 ➢ ( Τ 1 3 Stag – Τ 2 3 Hare, Τ 1 3 Stag – Τ 2 3 Hare) 1 1 1 1 o Total reward = 3 ∗ 3 ∗ 8 + 1 − 3 ∗ 3 ∗ 2 ∈ (2,8) • Price of stability? Price of anarchy? CSC304 - Nisarg Shah 5

Revisiting Prisoner’s Dilemma John Stay Silent Betray Sam Stay Silent (-1 , -1) (-3 , 0) Betray (0 , -3) (-2 , -2) • Min total cost = 1 + 1 = 2 • Only equilibrium: ➢ (Betray, Betray) : Total cost = 2 + 2 = 4 • Price of stability? Price of anarchy? CSC304 - Nisarg Shah 6

Cost Sharing Game • 𝑜 players on directed weighted graph 𝐻 • Player 𝑗 𝑡 1 𝑡 2 ➢ Wants to go from 𝑡 𝑗 to 𝑢 𝑗 1 1 ➢ Strategy set 𝑇 𝑗 = {directed 𝑡 𝑗 → 𝑢 𝑗 paths} ➢ Denote his chosen path by 𝑄 𝑗 ∈ 𝑇 𝑗 10 10 10 • Each edge 𝑓 has cost 𝑑 𝑓 (weight) ➢ Cost is split among all players taking edge 𝑓 1 1 ➢ That is, among all players 𝑗 with 𝑓 ∈ 𝑄 𝑗 𝑢 1 𝑢 2 CSC304 - Nisarg Shah 7

Cost Sharing Game • Given strategy profile 𝑄 , cost 𝑑 𝑗 𝑄 to player 𝑗 is sum of his costs for edges 𝑓 ∈ 𝑄 𝑗 𝑡 1 𝑡 2 • Social cost 𝐷 𝑄 = σ 𝑗 𝑑 𝑗 𝑄 1 1 • Note: 𝐷 𝑄 = σ 𝑓∈𝐹 𝑄 𝑑 𝑓 , where… 10 10 10 ➢ 𝐹(𝑄) ={edges taken in 𝑄 by at least one player} 1 1 ➢ Why? 𝑢 1 𝑢 2 CSC304 - Nisarg Shah 8

Cost Sharing Game • In the example on the right: ➢ What if both players take direct paths? ➢ What if both take middle paths? 𝑡 1 𝑡 2 ➢ What if one player takes direct path and the 1 1 other takes middle path? • Pure Nash equilibria? 10 10 10 1 1 𝑢 1 𝑢 2 CSC304 - Nisarg Shah 9

Cost Sharing: Simple Example • Example on the right: 𝑜 players s • Two pure NE ➢ All taking the n-edge: social cost = 𝑜 𝑜 1 ➢ All taking the 1-edge: social cost = 1 o Also the social optimum • Price of stability: 1 t • Price of anarchy: 𝑜 ➢ We can show that price of anarchy ≤ 𝑜 in every cost-sharing game! CSC304 - Nisarg Shah 10

Cost Sharing: PoA • Theorem: The price of anarchy of a cost sharing game is at most 𝑜 . • Proof: ∗ , 𝑄 ∗ , … , 𝑄 ∗ ) , in which ➢ Suppose the social optimum is (𝑄 𝑜 1 2 ∗ . the cost to player 𝑗 is 𝑑 𝑗 ➢ Take any NE with cost 𝑑 𝑗 to player 𝑗 . ′ be his cost if he switches to 𝑄 𝑗 ∗ . ➢ Let 𝑑 𝑗 ′ ≥ 𝑑 𝑗 ➢ NE ⇒ 𝑑 𝑗 (Why?) ′ ≤ 𝑜 ⋅ 𝑑 𝑗 ∗ (Why?) ➢ But : 𝑑 𝑗 ∗ for each 𝑗 ⇒ no worse than 𝑜 × optimum ➢ 𝑑 𝑗 ≤ 𝑜 ⋅ 𝑑 𝑗 ∎ CSC304 - Nisarg Shah 11

Cost Sharing • Price of anarchy ➢ Every cost-sharing game: PoA ≤ 𝑜 ➢ Example game with PoA = 𝑜 ➢ Bound of 𝑜 is tight. • Price of stability? ➢ In the previous game, it was 1 . ➢ In general, it can be higher. How high? ➢ We’ll answer this after a short detour. CSC304 - Nisarg Shah 12

Cost Sharing • Nash’s theorem shows existence of B 10 a mixed NE. ➢ Pure NE may not always exist in 20 A 12 general. C 7 60 32 • But in both cost-sharing games we E D saw, there was a PNE. 10 players: 𝐹 → 𝐷 ➢ What about a more complex 27 players: 𝐶 → 𝐸 game like the one on the right? 19 players: 𝐷 → 𝐸 CSC304 - Nisarg Shah 13

Good News • Theorem: Every cost-sharing game have a pure Nash equilibrium. • Proof: ➢ Via “potential function” argument CSC304 - Nisarg Shah 14

Step 1: Define Potential Fn • Potential function: Φ ∶ ς 𝑗 𝑇 𝑗 → ℝ + ➢ This is a function such that for every pure strategy profile ′ of 𝑗 , 𝑄 = 𝑄 1 ,… , 𝑄 𝑜 , player 𝑗 , and strategy 𝑄 𝑗 ′ , 𝑄 −𝑗 − 𝑑 𝑗 𝑄 = Φ 𝑄 𝑗 ′ , 𝑄 −𝑗 − Φ 𝑄 𝑑 𝑗 𝑄 𝑗 ➢ When a single player 𝑗 changes her strategy, the change in potential function equals the change in cost to 𝑗 ! ➢ In contrast, the change in the social cost 𝐷 equals the total change in cost to all players. o Hence, the social cost will often not be a valid potential function. CSC304 - Nisarg Shah 15

Step 2: Potential F n → pure Nash Eq • A potential function exists ⇒ a pure NE exists. ➢ Consider a 𝑄 that minimizes the potential function. ➢ Deviation by any single player 𝑗 can only (weakly) increase the potential function. ➢ But change in potential function = change in cost to 𝑗 . ➢ Hence, there is no beneficial deviation for any player. • Hence, every pure strategy profile minimizing the potential function is a pure Nash equilibrium. CSC304 - Nisarg Shah 16

Step 3: Potential F n for Cost-Sharing • Recall: 𝐹(𝑄) = {edges taken in 𝑄 by at least one player} • Let 𝑜 𝑓 (𝑄) be the number of players taking 𝑓 in 𝑄 𝑜 𝑓 (𝑄) 𝑑 𝑓 Φ 𝑄 = ෍ ෍ 𝑙 𝑙=1 𝑓∈𝐹(𝑄) • Note: The cost of edge 𝑓 to each player taking 𝑓 is 𝑑 𝑓 /𝑜 𝑓 (𝑄) . But the potential function includes all fractions: 𝑑 𝑓 /1 , 𝑑 𝑓 /2, …, 𝑑 𝑓 /𝑜 𝑓 𝑄 . CSC304 - Nisarg Shah 17

Step 3: Potential F n for Cost-Sharing 𝑜 𝑓 (𝑄) 𝑑 𝑓 Φ 𝑄 = ෍ ෍ 𝑙 𝑙=1 𝑓∈𝐹(𝑄) • Why is this a potential function? 𝑑 𝑓 ➢ If a player changes path, he pays 𝑜 𝑓 𝑄 +1 for each new 𝑑 𝑔 edge 𝑓 , gets back 𝑜 𝑔 𝑄 for each old edge 𝑔 . ➢ This is precisely the change in the potential function too. ➢ So Δ𝑑 𝑗 = ΔΦ . ∎ CSC304 - Nisarg Shah 18

Potential Minimizing Eq. • Minimizing the potential function gives some pure Nash equilibrium ➢ Is this equilibrium special? Yes! • Recall that the price of anarchy can be up to 𝑜 . ➢ That is, the worst Nash equilibrium can be up to 𝑜 times worse than the social optimum. • A potential-minimizing pure Nash equilibrium is better! CSC304 - Nisarg Shah 19

Potential Minimizing Eq. 𝑜 1 𝑜 𝑓 (𝑄) 𝑑 𝑓 ෍ 𝑑 𝑓 ≤ ≤ ෍ 𝑑 𝑓 ∗ ෍ Φ 𝑄 = ෍ ෍ 𝑙 𝑙 𝑙=1 𝑙=1 𝑓∈𝐹(𝑄) 𝑓∈𝐹(𝑄) 𝑓∈𝐹(𝑄) Social cost Harmonic function 𝐼(𝑜) ∀𝑄, 𝐷 𝑄 ≤ Φ 𝑄 ≤ 𝐷 𝑄 ∗ 𝐼 𝑜 𝑜 = σ 𝑙=1 1/𝑜 = 𝑃(log𝑜) 𝐷 𝑄 ∗ ≤ Φ 𝑄 ∗ ≤ Φ 𝑃𝑄𝑈 ≤ 𝐷 𝑃𝑄𝑈 ∗ 𝐼(𝑜) Potential minimizing eq. Social optimum CSC304 - Nisarg Shah 20

Potential Minimizing Eq. • Potential-minimizing PNE is 𝑃(log 𝑜) -approximation to the social optimum. • Thus, in every cost-sharing game, the price of stability is 𝑃 log𝑜 . ➢ Compare to the price of anarchy, which can be 𝑜 CSC304 - Nisarg Shah 21

Congestion Games • Generalize cost sharing games • 𝑜 players, 𝑛 resources (e.g., edges) • Each player 𝑗 chooses a set of resources 𝑄 𝑗 (e.g., 𝑡 𝑗 → 𝑢 𝑗 paths) • When 𝑜 𝑘 player use resource 𝑘 , each of them get a cost 𝑔 𝑘 (𝑜 𝑘 ) • Cost to player is the sum of costs of resources used CSC304 - Nisarg Shah 22

Congestion Games • Theorem [Rosenthal 1973]: Every congestion game is a potential game. • Potential function: 𝑜 𝑘 𝑄 Φ 𝑄 = 𝑔 𝑘 𝑙 ෍ ෍ 𝑙=1 𝑘∈𝐹(𝑄) • Theorem [Monderer and Shapley 1996]: Every potential game is equivalent to a congestion game. CSC304 - Nisarg Shah 23

Potential Functions • Potential functions are useful for deriving various results ➢ E.g., used for analyzing amortized complexity of algorithms • Bad news: Finding a potential function that works may be hard. CSC304 - Nisarg Shah 24

The Braess ’ Paradox • In cost sharing, 𝑔 𝑘 is decreasing ➢ The more people use a resource, the less the cost to each. • 𝑔 𝑘 can also be increasing ➢ Road network, each player going from home to work ➢ Uses a sequence of roads ➢ The more people on a road, the greater the congestion, the greater the delay (cost) • Can lead to unintuitive phenomena CSC304 - Nisarg Shah 25