CSCI 3210: Computational Game Theory Graphical Games Ref: [AGT] Ch - PDF document

4/16/18 CSCI 3210: Computational Game Theory Graphical Games Ref: [AGT] Ch 7 https://www.cis.upenn.edu/~mkearns/papers/agt-kearns.pdf Mohammad T . Irfan Email: mirfan@bowdoin.edu Web: www.bowdoin.edu/~mirfan Course Website: www.bowdoin.edu/~mirfan/CSCI-3210.html Usual games vs. graphical games u Usual games u The games we've seen so far u Representation size: O( n m n ) [normal form] u n = # of players, m = # of actions u A player's payoff is specified for each joint action 1

4/16/18 Usual games vs. graphical games u Graphical games u There's a graph/network u A player's payoff is directly determined by its neighbors' actions as well as its own action u Representation size (in normal form): O( n m d+1 ) + size of the graph u d is the max degree of a node Example: threshold game of complements u Binary actions: {0, 1} u Player (node) i chooses 1 if at least t i neighbors choose 1 u Player i chooses 0, if less than t i neighbors choose 1 A pure-strategy NE when all t i ’s are 2 4 2

4/16/18 Example: threshold game of complements u Another possible NE u Can we turn any of the 0s into 1 and still maintain NE? Another NE when all t i ’s are 2 5 How to specify such a game? u Graph/network u Players: n players, each a node u Actions: binary actions {0, 1} u Player i 's action, a i {0, 1} ∈ u Payoffs: u Player (node) i chooses 1 if at least t i neighbors choose 1 u Player i chooses 0, if less than t i neighbors choose 1 " % " % u Player i 's payoff function = $ ∑ ' a i $ a j ' '− t i + ε $ $ ' # # & & j :neighbor of i u What is the representation size here? 3

4/16/18 Representation size (comparison) u Games in normal form u O( n m n ) u Graphical games in normal form u O( n m d+1 ) + size of the graph Increasing size u Each node has a payoff matrix u Great when d << n u Graphical games in parametric form u O( n ) + size of the graph Pure-strategy NE computation In tree graphical games 4

4/16/18 Problem specification u Input: A graphical game in normal form u A tree graph u Each node/player's payoff matrix is specified w.r.t. the neighborhood joint actions u Here, neighborhood contains the player itself u Want: all pure-strategy NE u Variant: want one pure-strategy NE TreeNash algorithm u A two-pass algorithm u Message passing using dynamic programming u First pass: explore "feasible" pure strategies u Second pass: find pure-strategy NE 5

4/16/18 TreeNash: downstream pass Upstream( V ) u T(v, w) : Message that node V sends down to W , parameterized by all possible pure strategies v and w of L U 2 V and W , resp. U 1 u T(v, w) = 1 iff there exists an U k “upstream NE” from V , consistent with v and w u The node V and all the nodes upstream from V are "happy" (W may be unhappy) u T(v, w) = 0 o.w. T(v,w) v w OK? e.g. 0 0 1 0 1 0 1 0 1 1 1 1 TreeNash: downstream pass (cont...) Upstream( V ) u T(v, w) = 1 iff there exist actions u 1 , u 2 , ... of V 's parents U 1 , U 2 , ... such that L U 2 T( u i , v ) = 1, for all parents U i of V 1. U 1 Every parent U i to V : U k We're OK playing u i if you play v v is node V 's BR when node W plays 2. w and parents play u 1 , u 2 , ... V to W : T(v,w) I'm OK playing v if you play w Such a ( u 1 , u 2 , ... ) is a witness vector to T(v, w) = 1 6

4/16/18 TreeNash: upstream pass Upstream( V ) u W commands to each parent V: I'm playing w , you play v L u Will lead to a NE, as per the tables U 2 u V gets the command v , U 1 checks witness(es) to T(v,w) = 1, U k commands parents U 1 , U 2 , ... to play as per the witness vector u Commands propagate upward Play(w,v) Illustration of TreeNash algorithm witness vector L 2 L 1 u Threshold games of complement u t i = ½ | N i | (0,0) { } (0,0) { } (1,1) { } u Want: all pure-strategy NE (1,1) { } L 3 U 1 u Downstream pass (0,0) {(0,0)} (0,0) { } (1,0) {(1,1)} u O ( n d 2 d ) or O ( n 2 d ) with better d.s. (1,1) { } (0,1) {(0,0)} u Reason: T(u 1 , v) = 1 iff there exist (1,1) {(1,1)} L 4 actions x 1 , x 2 , ... of U 1 's parents X 1 , (0,0) {(0,0), (1,0)} (0,0) { } X 2 , ... such that (1)... and (2)... (0,1) {(0,0)} (1,1) { } (1,0) {(1,1)} u Existence check: O(2 d ) (1,1) {(0,1), (1,1)} (0) {(0,0)} (1) {(0,1), (1,0), (1,1)} 7

4/16/18 Illustration (cont…) L 2 L 1 0 u Upstream pass 0 u Nodes send command upward (0,0) { } 0 (0,0) { } (e.g. I’m playing 1, you play 0) (1,1) { } (1,1) { } L 3 u Each pure-strategy NE in O ( n d ) U 1 0 (0,0) {(0,0)} (0,0) { } (1,0) {(1,1)} 0 (1,1) { } (0,1) {(0,0)} 1 (1,1) {(1,1)} L 4 (0,0) {(0,0), (1,0)} (0,0) { } (0,1) {(0,0)} (1,1) { } (1,0) {(1,1)} 1 (1,1) {(0,1), (1,1)} (0) {(0,0)} (1) {(0,1), (1,0), (1,1)} Illustration (cont…) L 2 L 1 0 0 u Upstream pass 0 0 u Nodes send command upward (0,0) { } 1 0 1 (0,0) { } (e.g. I’m playing 1, you play 0) (1,1) { } (1,1) { } L 3 u Each pure-strategy NE in O ( n d ) U 1 0 0 (0,0) {(0,0)} (0,0) { } (1,0) {(1,1)} 1 (1,1) { } (0,1) {(0,0)} 1 (1,1) {(1,1)} L 4 (0,0) {(0,0), (1,0)} (0,0) { } (0,1) {(0,0)} (1,1) { } (1,0) {(1,1)} 1 (1,1) {(0,1), (1,1)} (0) {(0,0)} (1) {(0,1), (1,0), (1,1)} 8

4/16/18 Illustration (cont…) L 2 L 1 1 0 0 u Upstream pass 1 0 0 u Nodes send command upward (0,0) { } 0 1 1 (0,0) { } (e.g. I’m playing 1, you play 0) (1,1) { } (1,1) { } L 3 u Each pure-strategy NE in O ( n d ) U 1 0 1 0 (0,0) {(0,0)} (0,0) { } (1,0) {(1,1)} 1 (1,1) { } (0,1) {(0,0)} 1 (1,1) {(1,1)} L 4 (0,0) {(0,0), (1,0)} (0,0) { } (0,1) {(0,0)} (1,1) { } (1,0) {(1,1)} 1 (1,1) {(0,1), (1,1)} (0) {(0,0)} (1) {(0,1), (1,0), (1,1)} Running time u Total running time to compute one pure- strategy NE u O ( n 2 d + n d ) u Each additional pure-strategy NE: O( n d ) 9

4/16/18 Mixed-strategy NE u Binary action u Probabilities: [0,1] u Message from U to V: May exist infinite number of probabilities (u,v) s.t. T(u,v) = 1 u Good news: These probabilities can be represented as unions of axis-aligned rectangles within [0,1]x[0,1] u Bad news: The number of rectangles exponentially blow up as we go down stream to root! u Result: exponential time TreeNash algorithm for MSNE Approximation algorithm ( ε -NE) u Joint mixed strategy such that nobody gains more than ε by unilateral deviation u Every player plays ε -best response u A strategy is ε -BR if it's not more than ε worse than the actual BR u p* is an ε -NE iff every player i 's expected payoff u i ( p i * , p -i * ) satisfies: * ) + ε ≥ max * ) * , p − i u i ( p i u i ( a i , p − i a i u 0-NE is what we call NE 10

4/16/18 Technique u Discretize the mixed strategy of every player .2 .4 .6 .8 0 1 τ 2 τ ... ... 0 1 Grid size = 1/ τ u How many grid points? Relation with ε ? u How does the grid size affect computation? u Different grid size for different players? Approximate TreeNash u T( v , w ): v is V 's probability in the grid, w is W 's prob. in grid Upstream( V ) u Size of T( v , w ) = (1/ τ ) 2 L u Algorithm remains the same U 2 u Running time: O( n (1/ τ 2 ) d ) U 1 U k u Reason: T(v, w) = 1 iff there exist grid pt u 1 , u 2 , ... of V 's parents U 1 , U 2 , ... such that (1)... and (2)... u Existence check (brute force): O((1/ τ 2 ) d ) u Question T(v,w) u Relation between τ and ε for ε -NE? 11

4/16/18 Kearns, Littman, Singh (2001) Link: http://www.cis.upenn.edu/~mkearns/ papers/graphgames.pdf Kearns, Littman, Singh (2001) k = # of parents u ... u Running time (downstream): O( n (1/ τ 2 ) d ) u Is it a polynomial-time algorithm? u Representation size: O( n m d+1 ) [here m = 2] 12

4/16/18 Luis Ortiz (2002) u For an ε -NE, we only require τ <= ε /(4 d ) u Calculate the running time of TreeNash u Link: http://www.cis.upenn.edu/~mkearns/ teaching/cgt/revised_approx_bnd.pdf Ortiz & self (AAAI 2017) u Discretize both probability space and payoff space u A different algorithm: polynomial in the input size and 1/ ε u Such algorithms are known as Fully Polynomial Time Approximation Scheme (FPTAS) u Link: http://www.bowdoin.edu/~mirfan/papers/ Ortiz_Irfan_AAAI17_Tractable_Algorithms_for _Approximate_Nash_Equilibria.pdf 13

4/16/18 Open Problems u Efficient algorithms for exact NE computation in trees [polynomial in n and 2 d is fine] u Or prove that it is PPAD-hard u Negative result: No 2-pass dynamic programming will succeed [Elkind et al. , 2006] 14