Today. Notes. The multiplicative weights framework. Quick Review: - PowerPoint PPT Presentation

Today. Notes. The multiplicative weights framework. Quick Review: experts framework/multiplicative weights algorithm Got to definition of Approximate Equilibrium for zero sum Finish: games. randomized multiplicative weights algorithm for experts framework. Equilibrium for two person games: using experts framework/MW algorithm. Expert’s framework. Infallible expert. Concept Alert. One of the expert’s is infallible! Your strategy? Choose any expert that has not made a mistake! n experts. How long to find perfect expert? Note. Every day, each offers a prediction. Maybe..never! Never see a mistake. Adversary: “Rain” or “Shine.” Better model? makes you want to look bad. Whose advise do you follow? ”You could have done so well”... How many mistakes could you make? Mistake Bound. but you didn’t! ha..ha! “The one who is correct most often.” (A) 1 Analysis of Algorithms: do as well as possible! Sort of. (B) 2 How well do you do? (C) log n (D) n − 1 Adversary designs setup to watch who you choose, and make that expert make a mistake. n − 1!

Back to mistake bound. Alg 2: find majority of the perfect Imperfect Experts How many mistakes could you make? (A) 1 Goal? Infallible Experts. (B) 2 Do as well as the best expert! Alg: Choose one of the perfect experts. (C) log n Algorithm. Suggestions? Mistake Bound: n − 1 (D) n − 1 Go with majority? Lower bound: adversary argument. At most log n ! Upper bound: every mistake finds fallible expert. Penalize inaccurate experts? When alg makes a mistake, Better Algorithm? Best expert is penalized the least. | “perfect” experts | drops by a factor of two. Making decision, not trying to find expert! Initially n perfect experts mistake → ≤ n / 2 perfect experts 1. Initially: w i = 1. Algorithm: Go with the majority of previously correct experts. mistake → ≤ n / 4 perfect experts 2. Predict with weighted majority of experts. . . What you would do anyway! . 3. w i → w i / 2 if wrong. mistake → ≤ 1 perfect expert ≥ 1 perfect expert → at most log n mistakes! Analysis: weighted majority Analysis: continued. Best Analysis? 1. Initially: w i = 1. Goal: Best expert makes m mistakes. � M n . � 3 1 2 m ≤ ∑ i w i ≤ 2. Predict with 4 Potential function: ∑ i w i . Initially n . m - best expert mistakes M algorithm mistakes. weighted 1 Two experts: A,B For best expert, b , w b ≥ 2 m . majority of � 3 � M n . 1 2 m ≤ Each mistake: experts. Bad example? 4 Take log of both sides. total weight of incorrect experts reduced by 3. w i → w i / 2 if Which is worse? factor of 1 − 1? − 2? 2 ? − m ≤ − M log ( 4 / 3 )+ logn . wrong. each incorrect expert weight multiplied by 1 (A) A right on even, B right on odd. 2 ! Solve for M . total weight decreases by (B) A right first half of days, B right second M ≤ ( m + logn ) / log ( 4 / 3 ) ≤ 2 . 4 ( m + log n ) factor of 1 2 ? factor of 3 4 ? Multiple by 1 − ε for incorrect experts... mistake → ≥ half weight with incorrect experts. Best expert peformance: T / 2 mistakes. � M n . ( 1 − ε ) m ≤ Mistake → potential function decreased by 3 � 4 . 1 − ε Pattern (A): T − 1 mistakes. 2 We have Massage... Factor of (almost) two worse! M ≤ 2 ( 1 + ε ) m + 2ln n � M � 3 1 2 m ≤ ∑ ε w i ≤ n . 4 Approaches a factor of two of best expert performance! i where M is number of algorithm mistakes.

Randomization!!!! Randomized analysis. Randomized algorithm Losses in [ 0 , 1 ] . Expert i loses ℓ t i ∈ [ 0 , 1 ] in round t. Better approach? 1. Initially w i = 1 for expert i . Some formulas: Use? 2. Choose expert i with prob w i W , W = ∑ i w i . Randomization! For ε ≤ 1 , x ∈ [ 0 , 1 ] , 3. w i ← w i ( 1 − ε ) ℓ t i ( 1 + ε ) x ≤ ( 1 + ε x ) That is, choose expert i with prob ∝ w i ( 1 − ε ) x ≤ ( 1 − ε x ) W ( t ) sum of w i at time t . W ( 0 ) = n Bad example: A,B,A,B,A... Best expert, b , loses L ∗ total. → W ( T ) ≥ w b ≥ ( 1 − ε ) L ∗ . For ε ∈ [ 0 , 1 2 ] , After a bit, A and B make nearly the same number of mistakes. w i ℓ t − ε − ε 2 ≤ ln ( 1 − ε ) ≤ − ε L t = ∑ i W expected loss of alg. in time t . i Choose each with approximately the same probabilty. ε − ε 2 ≤ ln ( 1 + ε ) ≤ ε Claim: W ( t + 1 ) ≤ W ( t )( 1 − ε L t ) Loss → weight loss. Make a mistake around 1 / 2 of the time. Proof Idea: ln ( 1 + x ) = x − x 2 2 + x 3 3 −··· Proof: W ( t + 1 ) ≤ ∑ = ∑ ( 1 − ε ℓ t w i ℓ t Best expert makes T / 2 mistakes. w i − ε ∑ i ) w i i i i i Rougly optimal! � � 1 − ε ∑ i w i ℓ t = ∑ i w i ∑ i w i i = W ( t )( 1 − ε L t ) Analysis Gains. Summary: multiplicative weights. Why so negative? ( 1 − ε ) L ∗ ≤ W ( T ) ≤ n ∏ t ( 1 − ε L t ) Framework: n experts, each loses different amount every day. Each day, each expert gives gain in [ 0 , 1 ] . Perfect Expert: log n mistakes. Take logs Multiplicative weights with ( 1 + ε ) g t i . ( L ∗ ) ln ( 1 − ε ) ≤ ln n + ∑ ln ( 1 − ε L t ) Imperfect Expert: best makes m mistakes. Use − ε − ε 2 ≤ ln ( 1 − ε ) ≤ − ε Deterministic Strategy: 2 ( 1 + ε ) m + log n G ≥ ( 1 − ε ) G ∗ − log n ε Real numbered losses: Best loses L ∗ total. − ( L ∗ )( ε + ε 2 ) ≤ ln n − ε ∑ L t ε where G ∗ is payoff of best expert. Randomized Strategy: ( 1 + ε ) L ∗ + log n And ε ∑ t L t ≤ ( 1 + ε ) L ∗ + ln n ε . Scaling: Strategy: Choose proportional to weights ∑ t L t is total expected loss of algorithm. Not [ 0 , 1 ] , say [ 0 , ρ ] . multiply weight by ( 1 − ε ) loss. Within ( 1 + ε ) ish of the best expert! L ≤ ( 1 + ε ) L ∗ + ρ log n Multiplicative weights framework! No factor of 2 loss! ε Applications next!

Two person zero sum games. Equilibrium. Best Response m × n payoff matrix A . Row mixed strategy: x = ( x 1 ,..., x m ) . Column goes first: Find y , where best row is not too low.. Column mixed strategy: y = ( y 1 ,..., y n ) . Equilibrium pair: ( x ∗ , y ∗ ) ? Payoff for strategy pair ( x , y ) : x ( x t Ay ) . R = max min p ( x , y ) = ( x ∗ ) t Ay ∗ = max y ( x ∗ ) t Ay = min x x t Ay ∗ . y p ( x , y ) = x t Ay Note: x can be ( 0 , 0 ,..., 1 ,... 0 ) . (No better column strategy, no better row strategy.) That is, Example: Roshambo. Value of R ? No row is better: � � � � Row goes first: min i A ( i ) · y = ( x ∗ ) t Ay ∗ . 1 = ∑ ∑ ∑ ∑ x i a i , j y j x i a i , j y j . Find x , where best column is not high. i j j i No column is better: max j ( A t ) ( j ) · x = ( x ∗ ) t Ay ∗ . Recall row minimizes, column maximizes. y ( x t Ay ) . C = min x max Equilibrium pair: ( x ∗ , y ∗ ) ? Agin: y of form ( 0 , 0 ,..., 1 ,... 0 ) . ( x ∗ ) t Ay ∗ = max Example: Roshambo. Value of C ? y ( x ∗ ) t Ay = min x x t Ay ∗ . (No better column strategy, no better row strategy.) 1 A ( i ) is i th row. Duality. Proof of Equilibrium. Proof of approximate equilibrium. x ( x t Ay ) . R = max min y Later. Still later... y ( x t Ay ) . C = min x max Aproximate equilibrium ... How? Weak Duality: R ≤ C . C ( x ) = max y x t Ay Proof: Better to go second. R ( y ) = min x x t Ay (A) Using geometry. At Equilibrium ( x ∗ , y ∗ ) , payoff v : (B) Using a fixed point theorem. Always: R ( y ) < C ( x ) row payoffs ( Ay ∗ ) all ≥ v = ⇒ R ≥ v . (C) Using multiplicative weights. Strategy pair: ( x , y ) column payoffs ( ( x ∗ ) t A ) all ≤ v = ⇒ v ≥ C . (D) By the skin of my teeth. Equilibrium: ( x , y ) = ⇒ R ≥ C R ( y ) = C ( x ) → C ( x ) − R ( y ) = 0. (C) ..and (D). Equilibrium = ⇒ R = C ! Not hard. Even easy. Still, head scratching happens. Approximate Equilibrium: C ( x ) − R ( y ) ≤ ε . Strong Duality: There is an equilibrium point! and R = C ! With R ( y ) < C ( x ) Doesn’t matter who plays first! → “Response y to x is within ε of best response” → “Response x to y is within ε of best response”