Linear Programming in Low Dimensions (most slides by Nati Srebro) - PowerPoint PPT Presentation

Linear Programming in Low Dimensions (most slides by Nati Srebro) September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Linear Programming + + + Maximize : c x c x c x 1 1 2 2 d d + + + ≤ Subject to : a x a x a x b 1 , 1 1 1 , 2 2 1 , d d 1 + + + ≤ a x a x a x b 2 , 1 1 2 , 2 2 2 , d d 2 + + + ≤ a x a x a x b n , 1 1 n , 2 2 n , d d n September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Linear Programming in 2D + Maximize : c x c y x y + ≤ Subject to : a x a y b 1 , x 1 , y 1 + ≤ a x a y b 2 , x 2 , y 2 + ≤ a x a y b n , x n , y n September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Linear Programming in 2D September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Non-unique solution September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

An Infeasible Linear Program September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

An Unbounded LP September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

An Unbounded LP September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Types of LPs • Unique optimum � Find the optimum • Optimal edge � Find an optimum • Unbounded � Find unbounded ray • Infeasible � Declare as unfeasible September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint If its not broken don ’ t fix it ! September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint Is a bad sign … . September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Summary O(1) • Check is optimum is feasible • Optimum is feasible: We ’ re fine, don ’ t do anything • Optimum isn ’ t feasible: O(n) Find optimum on new constraint (line) No feasible points on new constraint: LP isn ’ t feasible September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Incremental Algorithm • Choose two constraints and initialize the solution • Add new constraints one by one, keeping track of current optimum September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Initialization Find two constraints that together bound the LP September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Initialization • Choose the constraint h defined by a vector that is the closest to “ up ” September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Initialization • Choose the constraint h defined by a vector that is the closest to “ up ” • For every other constraint, check if it bounds the LP with h • If no constraint is good--- LP is unbounded or unfeasible because of parallel constraints September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Incremental Algorithm • Find two constraints that bound LP – If none exist, LP is unbounded O(n) • Add all other constraints one by one, keeping track of current O(n 2 ) optimum September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Adding a New Constraint O(1) • Check is optimum is feasible • Optimum is feasible: We ’ re fine, don ’ t do anything • Optimum isn ’ t feasible: O(n) Find optimum on new constraint (line) No feasible points on new constraint: LP isn ’ t feasible Maybe we only rarely have to update optimum ? September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

O(n) updates … September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

But … September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Use a random permutation ! Expected time spent updating. Let T i be the time spent at time i. � � n n n � � � = = Update at E T E [ T ] � � P ( ) O ( i ) i i � � round i = = = i 3 i 3 i 3 September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Probability of update at round i Fix first i constraints: Update only if the i th 2 constraint is one of the 2 P ≤ = i = i two defining constraints P − − non-initial 2 2 constraints September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Expected Run-Time Analysis Expectation is over algorithm randomness, not over input Expected time spent updating: � � n n n � � � = = Update at E T E [ T ] � � P ( ) O ( i ) i i � � round i = = = i 3 i 3 i 3 n � 2 ≤ = O ( i ) O ( n ) − i 2 = i 3 September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

What about d>2 ? • Incrementally add new constraints • Probability of update: d/(i-d) • On update: solve d-1 dimensional LP n � d ≤ + − − T ( d , n ) O ( dn ) T ( d 1 , i 1 ) − i d = + i d 1 = T ( d , n ) O ( d ! n ) September 11, 2003 Lecture 3: Linear Programming in Low Dimensions

Further results • O(d!n) is not optimal: – O(d 2 n + d 2 d!) [Clarkson] – n O( √ d) [Kalai, Matousek-Sharir-Welzl] – O(d 2 n + d O( √ d) ) [combined] • Same time for finding minimum enclosing ball of n points • First algorithms of this type were due to Meggido • Weakly polynomial-time algorithms known [Khachiyan,Karmarkar] September 11, 2003 Lecture 3: Linear Programming in Low Dimensions