CS/ECE/ISyE 524 Introduction to Optimization Spring 2017–18 3. Linear programs ❼ Review: linear algebra ❼ Geometrical intuition ❼ Standard form for LPs ❼ Example: transformation to standard form Laurent Lessard (www.laurentlessard.com)
Matrix basics A matrix is an array of numbers. A ∈ R m × n means that: a 11 a 1 n . . . . . ... . . A = ( m rows and n columns) . . a m 1 a mn . . . Two matrices can be multiplied if inner dimensions agree: n � ( m × p ) = where c ij = C ( m × n ) B A a ik b kj ( n × p ) k =1 Example: 1 2 1 · 4 + 2 · 8 1 · 3 + 2 · 9 20 21 � 4 � 3 = 3 4 = 3 · 4 + 4 · 8 3 · 3 + 4 · 9 44 45 8 9 5 6 5 · 4 + 6 · 8 5 · 3 + 6 · 9 68 69 3-2
Matrix basics A matrix is an array of numbers. A ∈ R m × n means that: a 11 a 1 n . . . . . ... . . A = ( m rows and n columns) . . a m 1 a mn . . . Two matrices can be multiplied if inner dimensions agree: n � ( m × p ) = where c ij = C ( m × n ) B A a ik b kj ( n × p ) k =1 Example: 1 2 1 · 4 + 2 · 8 1 · 3 + 2 · 9 20 21 � 4 � 3 = 3 4 = 3 · 4 + 4 · 8 3 · 3 + 4 · 9 44 45 8 9 5 6 5 · 4 + 6 · 8 5 · 3 + 6 · 9 68 69 3-3
Matrix basics Transpose : The transpose operator A T swaps rows and columns. If A ∈ R m × n then A T ∈ R n × m and ( A T ) ij = A ji . ❼ ( A T ) T = A ❼ ( AB ) T = B T A T A vector is a column matrix. We write x ∈ R n to mean that: x 1 . (a vector x ∈ R n is an n × 1 matrix) . x = . x n The transpose of a column vector is a row vector: x T = � � x 1 · · · x n (i.e. a 1 × n matrix) 3-4
Matrix basics Two vectors x , y ∈ R n can be multiplied together in two ways. Both are valid matrix multiplications: ❼ inner product : produces a scalar. y 1 . x T y = � � . · · · x 1 x n = x 1 y 1 + · · · + x n y n . y n Also called “dot product”. Often written x · y or � x , y � . ❼ outer product : produces an n × n matrix. x 1 x 1 y 1 x 1 y n . . . . . . xy T = ... . � � . . y 1 · · · y n = . . . x n x n y 1 x n y n . . . 3-5
Matrix basics ❼ Matrices and vectors can be stacked and combined to form bigger matrices as long as the dimensions agree. e.g. If � � x 1 , . . . , x m ∈ R n , then X = ∈ R m × n . x 1 x 2 x m . . . ❼ Matrices can also be concatenated in blocks. For example: � A � B if A , C have same number of columns, Y = C D A , B have same number of rows, etc. ❼ Matrix multiplication also works with block matrices! � A � � P � � AP + BQ � B = CP + DQ C D Q as long as A has as many columns as P has rows, etc. 3-6
Linear and affine functions ❼ A function f ( x 1 , . . . , x m ) is linear in the variables x 1 , . . . , x m if there exist constants a 1 , . . . , a m such that f ( x 1 , . . . , x m ) = a 1 x 1 + · · · + a m x m = a T x ❼ A function f ( x 1 , . . . , x m ) is affine in the variables x 1 , . . . , x m if there exist constants b , a 1 , . . . , a m such that f ( x 1 , . . . , x m ) = a 0 + a 1 x 1 + · · · + a m x m = a T x + b Examples: Some texts use “linear” to mean either one! 1. 3 x − y is linear in ( x , y ). 2. 2 xy + 1 is affine in x and y but not in ( x , y ). 3. x 2 + y 2 is not linear or affine. 3-7
Linear and affine functions Several linear or affine functions can be combined: a 11 x 1 + · · · + a 1 n x n + b 1 a 11 a 1 n x 1 b 1 . . . a 21 x 1 + · · · + a 2 n x n + b 2 . . . . ... . . . . = ⇒ + . . . . . . . . . . . . . a m 1 a mn x n b m . . . a m 1 x 1 + · · · + a mn x n + b m which can be written simply as Ax + b . Same definitions apply: ❼ A vector-valued function F ( x ) is linear in x if there exists a constant matrix A such that F ( x ) = Ax . ❼ A vector-valued function F ( x ) is affine in x if there exists a constant matrix A and vector b such that F ( x ) = Ax + b . 3-8
Geometry of affine equations ❼ The set of points x ∈ R n that satisfies a linear equation a 1 x 1 + · · · + a n x n = 0 (or a T x = 0) is called a hyperplane . The vector a is normal to the hyperplane. ❼ If the right-hand side is nonzero: a T x = b , the solution set is called an affine hyperplane , (it’s a shifted hyperplane). a Affine hyperplane in 2D Affine hyperplane in 3D 3-9
Geometry of affine equations ❼ The set of points x ∈ R n satisfying many linear equations a i 1 x 1 + · · · + a im x n = 0 for i = 1 , . . . , m (or Ax = 0) is called a subspace (the intersection of many hyperplanes). ❼ If the right-hand side is nonzero: Ax = b , the solution set is called an affine subspace , (it’s a shifted subspace). Intersections of affine hyperplanes are affine subspaces. 3-10
Geometry of affine equations The dimension of a subspace is the number of independent directions it contains. A line has dimension 1, a plane has dimension 2, and so on. Hyperplanes are subspaces! ❼ A hyperplane in R n is a subspace of dimension n − 1. ❼ The intersection of k hyperplanes has dimension at least n − k (“at least” because of potential redundancy). 3-11
Affine combinations If x , y ∈ R n , then the combination w = α x + (1 − α ) y for some α ∈ R is called an affine combination . x α x + (1 − α ) y y 0 If Ax = b and Ay = b , then Aw = b . So affine combinations of points in an (affine) subspace also belong to the subspace. 3-12
Affine combinations If x , y ∈ R n , then the combination w = α x + (1 − α ) y for some α ∈ R is called an affine combination . Equivalently: x y + α ( x − y ) y 0 If Ax = b and Ay = b , then Aw = b . So affine combinations of points in an (affine) subspace also belong to the subspace. 3-13
Convex combinations If x , y ∈ R n , then the combination w = α x + (1 − α ) y for some 0 ≤ α ≤ 1 is called a convex combination (for reasons we will learn later). It’s the line segment that connects x and y . x α x + (1 − α ) y y 0 3-14
Geometry of affine inequalities ❼ The set of points x ∈ R n that satisfies a linear inequality a 1 x 1 + · · · + a n x n ≤ b (or a T x ≤ b ) is called a halfspace . The vector a is normal to the halfspace and b shifts it. ❼ Define w = α x + (1 − α ) y where 0 ≤ α ≤ 1. If a T x ≤ b and a T y ≤ b , then a T w ≤ b . a Halfspace 3-15
Geometry of affine inequalities ❼ The set of points x ∈ R n satisfying many linear inequalities a i 1 x 1 + · · · + a in x n ≤ b i for i = 1 , . . . , m (or Ax ≤ b ) is called a polyhedron (the intersection of many halfspaces). Some sources use the term polytope instead. ❼ As before: let w = α x + (1 − α ) y where 0 ≤ α ≤ 1. If Ax ≤ b and Ay ≤ b , then Aw ≤ b . Intersections of halfspaces are polyhedra. 3-16
Solutions of an LP There are exactly three possible cases: 1. Model is infeasible : there is no x that satisfies all the constraints. infeasible (is the model correct?) 2. Model is feasible, but unbounded : the cost function can be arbitrarily improved. (forgot a constraint?) unbounded 3. Model has a solution which occurs on the boundary of the set. (there may be many solutions!) boundary 3-17
The linear program A linear program is an optimization model with: ❼ real-valued variables ( x ∈ R n ) ❼ affine objective function ( c T x + d ), can be min or max. ❼ constraints may be: ◮ affine equations ( Ax = b ) ◮ affine inequalities ( Ax ≤ b or Ax ≥ b ) ◮ combinations of the above ❼ individual variables may have: ◮ box constraints ( p ≤ x i , or x i ≤ q , or p ≤ x i ≤ q ) ◮ no constraints ( x i is unconstrained) There are many equivalent ways to express the same LP 3-18
Standard form ❼ Every LP can be put in the form: c T x maximize x ∈ R n subject to: Ax ≤ b x ≥ 0 ❼ This is called the standard form of a LP. 3-19
Back to Top Brass � T � � � 12 f max max 12 f + 9 s 9 s f , s f , s 4 2 4800 s.t. 4 f + 2 s ≤ 4800 = ⇒ � � 1 1 f 1750 s.t. ≤ f + s ≤ 1750 1 0 1000 s 0 ≤ f ≤ 1000 0 1 1500 0 ≤ s ≤ 1500 � � f ≥ 0 s This is in standard form, with: 4 2 4800 � 12 � � f � 1 1 1750 A = b = c = x = , , , 1 0 1000 9 s 0 1 1500 3-20
Transformation tricks 1. converting min to max or vice versa (take the negative): min f ( x ) = − max ( − f ( x )) x x 2. reversing inequalities (flip the sign): Ax ≤ b ⇐ ⇒ ( − A ) x ≥ ( − b ) 3. equalities to inequalities (double up): f ( x ) = 0 ⇐ ⇒ f ( x ) ≥ 0 and f ( x ) ≤ 0 4. inequalities to equalities (add slack): f ( x ) ≤ 0 ⇐ ⇒ f ( x ) + s = 0 and s ≥ 0 3-21
Transformation tricks 5. unbounded to bounded (add difference): x ∈ R ⇐ ⇒ u ≥ 0 , v ≥ 0 , and x = u − v 6. bounded to unbounded (convert to inequality): � � � � 1 q p ≤ x ≤ q ⇐ ⇒ x ≤ − 1 − p 7. bounded to nonnegative (shift the variable) p ≤ x ≤ q ⇐ ⇒ 0 ≤ ( x − p ) and ( x − p ) ≤ ( q − p ) 3-22
Recommend
More recommend
