CS 473: Algorithms Chandra Chekuri Ruta Mehta University of Illinois, Urbana-Champaign Fall 2016 Chandra & Ruta (UIUC) CS473 1 Fall 2016 1 / 39
CS 473: Algorithms, Fall 2016 Simplex and LP Duality Lecture 19 October 28, 2016 Chandra & Ruta (UIUC) CS473 2 Fall 2016 2 / 39
Outline Simplex: Intuition and Implementation Details Computing starting vertex: equivalent to solving an LP! Infeasibility, Unboundedness, and Degeneracy. Duality: Bounding the objective value through weak-duality Strong Duality, Cone view. Chandra & Ruta (UIUC) CS473 3 Fall 2016 3 / 39
Part I Recall Chandra & Ruta (UIUC) CS473 4 Fall 2016 4 / 39
Feasible Region and Convexity Canonical Form Given A ∈ R n × d , b ∈ R n × 1 and c ∈ R 1 × d , find x ∈ R d × 1 max : c · x s . t . Ax ≤ b Chandra & Ruta (UIUC) CS473 5 Fall 2016 5 / 39
Linear Inequalities Define a Polyhedron If � j a ij x j ≤ b i hold we equality, we say the constraint/hyperplane i is tight Chandra & Ruta (UIUC) CS473 6 Fall 2016 6 / 39
Vertex Solution Optimizing linear objective over a polyhedron ⇒ Vertex solution Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Vertex Solution Optimizing linear objective over a polyhedron ⇒ Vertex solution Basic Feasible Solution: feasible, and d linearly independent tight constraints. Chandra & Ruta (UIUC) CS473 7 Fall 2016 7 / 39
Summary Each linear constraint defines a halfspace . 1 Feasible region, which is an intersection of halfspaces, is a 2 convex polyhedron . Optimal value attained at a vertex of the polyhedron. 3 Chandra & Ruta (UIUC) CS473 8 Fall 2016 8 / 39
Part II Simplex Chandra & Ruta (UIUC) CS473 9 Fall 2016 9 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions Which neighbor to move to? When to stop? How much time does it take? Chandra & Ruta (UIUC) CS473 10 Fall 2016 10 / 39
Observations For Simplex Suppose we are at a non-optimal vertex ˆ x and optimal is x ∗ , then c · x ∗ > c · ˆ x . Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Observations For Simplex Suppose we are at a non-optimal vertex ˆ x and optimal is x ∗ , then c · x ∗ > c · ˆ x . x to x ∗ on the line joining How does (c · x) change as we move from ˆ the two? Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Observations For Simplex Suppose we are at a non-optimal vertex ˆ x and optimal is x ∗ , then c · x ∗ > c · ˆ x . x to x ∗ on the line joining How does (c · x) change as we move from ˆ the two? Strictly increases! Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Observations For Simplex Suppose we are at a non-optimal vertex ˆ x and optimal is x ∗ , then c · x ∗ > c · ˆ x . x to x ∗ on the line joining How does (c · x) change as we move from ˆ the two? Strictly increases! d = x ∗ − ˆ x is the direction from ˆ x to x ∗ . (c · d) = (c · x ∗ ) − (c · ˆ x) > 0 . In x = ˆ x + δ d , as δ goes from 0 to 1 , we move from ˆ x to x ∗ . c · x = c · ˆ x + δ (c · d) . Strictly increasing with δ ! Due to convexity, all of these are feasible points. Chandra & Ruta (UIUC) CS473 11 Fall 2016 11 / 39
Cone Definition Given a set of vectors D = { d 1 , . . . , d k } , the cone spanned by them is just their positive linear combinations, i.e., k � cone(D) = { d | d = λ i d i , where λ i ≥ 0 , ∀ i } i=1 � � � � � � � � � � � � Chandra & Ruta (UIUC) CS473 12 Fall 2016 12 / 39
Cone at a Vertex Let z 1 , . . . , z k be the neighboring vertices of ˆ x . And let d i = z i − ˆ x be the direction from ˆ x to z i . � � Lemma � � Any feasible direction of movement d from ˆ x is in the cone( { d 1 , . . . , d k } ) . � � � � � � � � Chandra & Ruta (UIUC) CS473 13 Fall 2016 13 / 39
Improving Direction Implies Improving Neighbor Lemma If d ∈ cone( { d 1 , . . . , d k } ) and (c · d) > 0 , then there exists d i such that (c · d i ) > 0 . Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 39
Improving Direction Implies Improving Neighbor Lemma If d ∈ cone( { d 1 , . . . , d k } ) and (c · d) > 0 , then there exists d i such that (c · d i ) > 0 . Proof. To the contrary suppose (c · d i ) ≤ 0 , ∀ i ≤ k . Since d is a positive linear combination of d i ’s, (c · � k (c · d) = i=1 λ i d i ) � k = i=1 λ i (c · d i ) ≤ 0 A contradiction! Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 39
Improving Direction Implies Improving Neighbor Lemma If d ∈ cone( { d 1 , . . . , d k } ) and (c · d) > 0 , then there exists d i such that (c · d i ) > 0 . Proof. To the contrary suppose (c · d i ) ≤ 0 , ∀ i ≤ k . Since d is a positive linear combination of d i ’s, (c · � k (c · d) = i=1 λ i d i ) � k = i=1 λ i (c · d i ) ≤ 0 A contradiction! Theorem If vertex ˆ x is not optimal then it has a neighbor where cost improves. Chandra & Ruta (UIUC) CS473 14 Fall 2016 14 / 39
How Many Neighbors a Vertex Has? Geometric view... A ∈ R n × d ( n > d ), b ∈ R n , the constraints are: Ax ≤ b Faces Vertex: 0-dimensional face. Edge: 1D face. . . . Hyperplane: (d − 1) D face. Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 39
How Many Neighbors a Vertex Has? Geometric view... A ∈ R n × d ( n > d ), b ∈ R n , the constraints are: Ax ≤ b Faces Vertex: 0-dimensional face. Edge: 1D face. . . . Hyperplane: (d − 1) D face. r linearly independent tight hyperplanes forms d − r dimensional face. Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 39
How Many Neighbors a Vertex Has? Geometric view... A ∈ R n × d ( n > d ), b ∈ R n , the constraints are: Ax ≤ b Faces Vertex: 0-dimensional face. Edge: 1D face. . . . Hyperplane: (d − 1) D face. r linearly independent tight hyperplanes forms d − r dimensional face. Vertices being of 0 D, d L.I. tight hyperplanes. Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 39
How Many Neighbors a Vertex Has? Geometric view... A ∈ R n × d ( n > d ), b ∈ R n , the In 2-dimension ( d = 2 ) constraints are: Ax ≤ b x 2 Faces Vertex: 0-dimensional face. 300 Edge: 1D face. . . . Hyperplane: (d − 1) D face. r linearly independent tight hyperplanes forms d − r dimensional face. Vertices being of 0 D, d L.I. x 1 200 tight hyperplanes. Chandra & Ruta (UIUC) CS473 15 Fall 2016 15 / 39
How Many Neighbors a Vertex Has? Geometric view... A ∈ R n × d ( n > d ), b ∈ R n , the In 3-dimension ( d = 3 ) constraints are: Ax ≤ b Faces Vertex: 0-dimensional face. �′ Edge: 1D face. . . . ② � � Hyperplane: (d − 1) D face. ① r linearly independent tight ③ constraints forms d − r dimensional face. Vertices (Basic feasible solution) has d L.I. tight constraints. image source: webpage of Prof. Forbes W. Lewis Chandra & Ruta (UIUC) CS473 16 Fall 2016 16 / 39
How Many Neighbors a Vertex Has? Geometry view... One neighbor per tight hyperplane. Therefore typically d . Suppose x ′ is a neighbor of ˆ x , then on the edge joining the �′ two d − 1 constraints are ② � � tight. ④ These d − 1 are also tight at ① ③ both ˆ x and x ′ . x One more constraints, say i , is tight at ˆ x . “Relaxing” i at ˆ x leads to x ′ . Chandra & Ruta (UIUC) CS473 17 Fall 2016 17 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. When to stop? When no neighbor with better objective value. Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 39
Simplex Algorithm Simplex: Vertex hoping algorithm Moves from a vertex to its neighboring vertex Questions + Answers Which neighbor to move to? One where objective value increases. When to stop? When no neighbor with better objective value. How much time does it take? At most d neighbors to consider in each step. Chandra & Ruta (UIUC) CS473 18 Fall 2016 18 / 39
Simplex in Higher Dimensions Simplex Algorithm Start at a vertex of the polytope. 1 Compare value of objective function at each of the d 2 “neighbors”. Move to neighbor that improves objective function, and repeat 3 step 2. If no improving neighbor, then stop. 4 Chandra & Ruta (UIUC) CS473 19 Fall 2016 19 / 39
Recommend
More recommend
