CSCI 3210: Computational Game Theory Linear Programming and - PDF document

2/14/18 CSCI 3210: Computational Game Theory Linear Programming and 2-Player Zero-Sum Games Ref: Wikipedia and [AGT] Ch 1 Mohammad T . Irfan Email: mirfan@bowdoin.edu Web: www.bowdoin.edu/~mirfan Course Website: www.bowdoin.edu/~mirfan/CSCI-3210.html 2-player zero-sum game u Prove that NE exists – in two ways 1. Nash's theorem Doesn't give an algorithm (why?) u 2. Linear programming u Gives an algorithm 1

2/14/18 Example: 2-player zero-sum game u Penalty kick game Goalkeeper Left Right (0.42) (0.58) Shooter 0.58, 0.95, Left (0.38) 0.42 0.05 0.93, 0.70, Right 0.07 0.30 (0.62) Example: 2-player zero-sum game u Assumption (wlog): sum of payoffs in each cell is 0 Column player L R L R Row player 2, -2 -1, 1 U 2 -1 U -3, 3 4, -4 D -3 4 D u More than 2 actions? u Need an algorithm 2

2/14/18 Linear Programming (LP) Will come back to game theory later Applications u Optimization u Production, machine scheduling, employee scheduling, supply chain management, etc. u Game theory u In general: optimization 3

2/14/18 LP 1. Variables (or decision variables) u We can choose the values of these variables u What's the goal? u What values can we choose? 2. Objective function (What's the goal?) u Minimization or maximization u Must be linear in the variables 3. Constraints (What values?) u Restricts the values of choice variables u Must be linear in the variables Example: LP formulation & geometric interpretation u One is planning his day-to-day life. Outside of 10 hours of sleep every day, he wants to set aside a few hours for studying and a few hours for connecting with friends. u Gets 10 units/hr of payoff from study and 20 units/hr of payoff from connecting with friends. u Must study at least 6 hours every day. Also, feels guilty if spends more than 6 hours with friends. u How should he allocate time optimally? u Variables? u Objective function? u Constraints? 4

2/14/18 14 One of the vertices (black dots) will give the optimal solution x2 6 Feasible region (0,0) 6 x1 14 Note: x1, x2 >= 0: white region Example: infeasible LP u Want to study at least 10 hours/day and do other activities for at least 5 hours/day. How to allocate time? 5

2/14/18 Example: more var. & constraints u Gets 15 units/hr of payoff for studying up to 3 hours and 10 units/hr of payoff after 3 hours of studying (basically, brain slows down). Also gets 20 units/hr of payoff from connecting with friends. u Wants at least 6 hours of study u Wants at most 6 hours of time with friends Example: unbounded LP u A tennis player is making a plan for practicing service and volley. She gets a payoff of 10 from every service and 5 from every volley. u She wants to practice service at least 100 times a day and doesn't want to practice volleys more than 500 times a day. What's her optimal plan? 6

2/14/18 Matrix algebra u Images from this tutorial: http://www.intmath.com/matrices- determinants/3-matrices.php u 4x1 matrix (AKA vector) u 3x3 matrix Matrix multiplication u 2x3 matrix multiplied by 3x2 matrix must match u Result is a 2x2 matrix 7

2/14/18 Transpose of matrix u Transpose operator: superscript T ! $ 1 4 # & A = 2 5 # & # 3 6 & " % ! $ 1 2 3 A T = # & # & 4 5 6 " % u (A B) T = B T A T 8

2/14/18 Algorithms for solving LP u Simplex (Dantzig, 1947) u Worst case exponential time u Practically fast u Ellipsoid (Khachiyan, 1979) u O(n 4 L) for n variables and L input bits u Pseudo-polynomial u Karmarkar's algorithm (Karmarkar, 1984) u O(n 3.5 L) for n variables and L input bits u Pseudo-polynomial, but breakthrough for practical reasons u Open problem: strongly polynomial algorithm? LP Duality (von Neumann, 1947) u Interview with Dantzig u http://www.personal.psu.edu/ecb5/Courses/ M475W/WeeklyReadings/Week%2015/ An_Interview_with_George_Dantzig.pdf u If the "primal" LP is maximization, its "dual" is minimization and vice versa. u Every variable of the primal LP leads to a constraint in the dual LP and every constraint of the primal LP leads to a variable in the dual LP . u Dual of dual is primal. 9

2/14/18 Definition of dual LP Source: Applied Mathematical Programming book Primal Maximize c T x subject to: A x <= b x >= 0 Dual Minimize b T y subject to: A T y >= c y >= 0 Example: LP duality u How many Bowdoin logs and chocolate cakes should Thorne make to maximize its revenue? Derive primal and dual LP u Objective function. Each log has a satisfaction of 10 (or price of $10), each cake 5. u Constraints. For both desserts, the chef needs to use an oven, a food processor, and a boiler. Processing time/log Processing time/cake Total available time Oven 5 min 1 min 85 min Food processor 1 min 10 min 300 min Boiler 4 min 6 min 120 min 10

2/14/18 Dual: intuition u Moulton wants to borrow Thorne's equipment for a day. Thorne wants to charge "reasonable" prices $y1/min, $y2/min, and $y3/min for the 3 equipment, resp. u Dual objective: minimize total cost of renting u Dual constraints: recuperate lost payoff for each dessert Daily planner example Primal LP Dual LP Maximize 10 x1 + 20 x2 Subject to ? x1 >= 6 (or, -x1 <= -6) x2 <= 6 x1 + x2 <= 14 x1, x2 >= 0 Work out the solutions by hand. What's the dual interpretation? 11

2/14/18 Complementary slackness u primal constraint non-binding (not equal) => corresponding dual variable = 0 u Similar condition holds for dual constr. & primal var. u The reverse implication may not hold Weak duality theorem Dual LP Minimize ... Increasing objective function Gap? Primal LP Maximize ... 12

2/14/18 Weak duality theorem u Any feasible solution of the dual LP (minimization) gives an upper bound on the optimal solution of the primal LP (maximization). u Proof u Any feasible solution of the primal LP (maximization) is a lower bound on the optimal solution of the dual LP (minimization). Increasing objective function Dual LP (min) Gap? Primal LP (max) Weak duality theorem u Implications u Primal unbounded è Dual infeasible u Dual unbounded è Primal infeasible u Both primal and dual may be infeasible (although not implied by this theorem) Increasing objective function Dual LP (min) Gap? Primal LP (max) 13

2/14/18 Strong duality theorem u If the primal LP has a finite optimal solution, then so does the dual LP . Moreover, these two optimal solutions have the same objective function value. u In other words, if either the primal or the dual LP has a finite optimal solution, the gap between them is 0. 2-player zero-sum game Algorithm via LP duality 14

2/14/18 Example: 2-player zero-sum game u Assumption (wlog): sum of payoffs in each cell is 0 Column player L R L R Row player 2, -2 -1, 1 U 2 -1 U -3, 3 4, -4 D -3 4 D Matrix A Row player u How much ( v r ) can row guarantee from col.? u Want largest v r possible u Row: choose mixed strategy p (vector of prob.) to maximize v r u Expected payoff of row when of col. plays j = ( p T A ) j = Σ i ( p i A i,j ) 15

2/14/18 Row player's LP Row player's thought process: maximize my guaranteed payoff v from column player knowing that column player will always minimize her payout to me (in v r = max v other words, v is the least for any subject to action j of column player). ∑ p i A i , j ≥ v , for each action j of column player i ∑ p i = 1 i p i ≥ 0, for each action i of row player Column player u How little ( v c ) can col. player pay to row? u Choose mixed strategy q (vector of probabilities) to minimize v c u Expected payoff of row player for action i = ( Aq ) i = Σ j ( A i,j q j ) Col. player's thought process: minimize my guaranteed payout u to row player u LP knowing that row player will v c = min u choose to maximize her payoff. subject to ∑ A i , j q j ≤ u , for each action i of row player j ∑ q j = 1 j q j ≥ 0, for each action j of column player 16

2/14/18 Column player's LP Col. player's thought process: minimize my guaranteed payout u v c = min u to row player knowing that row player will subject to choose to maximize her payoff. ∑ A i , j q j ≤ u , for each action i of row player j ∑ q j = 1 j q j ≥ 0, for each action j of column player Minimax Theorem u At an equilibrium, v r = v c u Proof: 1. The two LPs are duals of each other. 2. Primal LP has a finite optimal solution (it's feasible + bounded). 3. By the strong duality theorem, v r = v c . u Another proof: 1. Let v* be row player's payoff at a NE. 2. v r <= v *, because is v r is row player's guaranteed payoff and v * cannot be lower than that. 3. By assumption of NE, column player will not give row player more than v r . So, v r = v *. SImilarly, v c = v *. u This quantity v c or v r is known as the value of the game ( v* ) 17

2/14/18 More LP duality example Vertex cover problem Vertex cover problem u Given a graph, select the minimum number of nodes such that at least one endpoint of every edge is selected. 1 2 0 3 4 7 8 9 5 6 u Answer? 18

2/14/18 Optimization u x t is 1 if node t is picked and 0 otherwise u Is the following a usual linear program? Minimize Σ t x t Subject to x u + x v >= 1, for each edge ( u , v ) x t ε {0, 1}, for each node t Integer linear program (ILP) In general "NP-hard" to solve LP relaxation Minimize Σ t x t Subject to x u + x v >= 1, for each edge ( u , v ) x t >= 0, for each node t Approximation algorithm: Rounding the LP solution gives a 2-approximation 19