Rank and Range Marco Chiarandini Department of Mathematics & - PowerPoint PPT Presentation

DM559 Linear and Integer Programming Lecture 6 Rank and Range Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

Numerical Issues in Python Rank Outline Range 1. Numerical Issues in Python 2. Rank 3. Range 2

Numerical Issues in Python Rank Outline Range 1. Numerical Issues in Python 2. Rank 3. Range 3

Numerical Issues in Python Rank Exercise Range Solve the following system of linear equations:  x + z = 1  3 x + 4 y + z = 1 + 4 y − 2 z = − 2  � � � � 4 1 3 0 � � � � det ( A ) = 1 � + 1 � = 0 � � � � 4 − 2 1 − 2 � � Hence we cannot solve by inverse nor by Cramer’s rule. We proceed by Gaussian elimination:  1 0 1 1   1 0 1 1  x = 1 − t  → y = − 1 2 + 1 3 4 1 1 0 1 − 1 / 2 − 1 / 2 , ∀ t ∈ R 2 t    0 4 − 2 − 2 0 0 0 0 z = t       x 1 − 1  = − 1  + 1  t , ∀ t ∈ R y infinitely many solutions    2 2 z 0 1 4

Numerical Issues in Python Rank Range In Python: ✞ ☎ import numpy as np r1=np.array([1,0,1]) r2=np.array([3,4,1]) r3=r2 − 3 ∗ r1 A=np.vstack([r1,r2,r3]) ✝ ✆ ✞ ☎ ✞ ☎ np.linalg.det(A) np.linalg.solve(A, [1,1, − 2]) # 1.3322676295501906e − 15 # array([ 0., 0., 1.]) ✝ ✆ np.dot(np.linalg.inv(A), [1,1, − 2]) # array([ − 1., 0., 1.]) np.dot(np.linalg.inv(A),A) # array([[ 0., 0., − 1.],[ 0., 1., 0.],[ 0., 0., 1.]]) ✝ ✆ Hence, Python for numerical reasons does not recognize the determinant to be null and solves the system returning only one particular solution. Knoweldge of the theory of linear algebra is important to avoid mistakes! ✞ ☎ import sympy as sy or you can carry out the row operations M=sy.Matrix(A) yourself M.rref() ✝ ✆ 5

Numerical Issues in Python Rank Outline Range 1. Numerical Issues in Python 2. Rank 3. Range 6

Numerical Issues in Python Rank Rank Range • Synthesis of what we have seen so far under the light of two new concepts: rank and range of a matrix • We saw that: every matrix is row-equivalent to a matrix in reduced row echelon form. Definition (Rank of Matrix) The rank of a matrix A , rank ( A ) , is • the number of non-zero rows, or equivalently • the number of leading ones in a row echelon matrix obtained from A by elementary row operations. � For an m × n matrix A , rank A ≤ min { m , n } , where min { m , n } denotes the smaller of the two integers m and n . 7

Numerical Issues in Python Rank Range Example   1 2 1 1 M = 2 3 0 5   3 5 1 6   R ′ 2 = R 2 − 2 R 1   R ′   1 2 1 1 1 2 1 1 2 = − R 2 1 2 1 1 R ′ R ′ 3 = R 3 − 3 R 1 3 = R 3 − R 2 2 3 0 5 − − − − − − − → 0 − 1 − 2 3 − − − − − − − → 0 1 2 − 3       3 5 1 6 0 − 1 − 2 3 0 0 0 0 � rank ( M ) = 2 8

Numerical Issues in Python Rank Extension of the main theorem Range Theorem If A is an n × n matrix, then the following statements are equivalent: 1. A is invertible 2. A x = b has a unique solution for any b ∈ R 3. A x = 0 has only the trivial solution, x = 0 4. the reduced row echelon form of A is I . 5. | A | � = 0 6. the rank of A is n 9

Numerical Issues in Python Rank Rank and Systems of Linear Equations Range x + 2 y + z = 1 2 x + 3 y = 5 3 x + 5 y + z = 4   R ′ 2 = R 2 − 2 R 1   R ′   1 2 1 1 1 2 1 1 2 = − R 2 1 2 1 1 R ′ R ′ 3 = R 3 − 3 R 1 3 = R 3 − R 2 2 3 0 5 − − − − − − − → 0 − 1 − 2 3 − − − − − − − → 0 1 2 − 3       3 5 1 4 0 − 1 − 2 1 0 0 0 − 2 x + 2 y + z = 1 The last row is of the type 0 = a , a � = 0, that is, x + 2 z = − 3 the augmenting matrix has a leading one in the 0 x + 0 y + 0 z = − 2 last column It is inconsistent! rank ( A ) = 2 � = rank ( A | b ) = 3 1. A system A x = b is consistent if and only if the rank of the augmented matrix is precisely the same as the rank of the matrix A . 10

Numerical Issues in Python Rank Range 2. If an m × n matrix A has rank m , the system of linear equations, A x = b , will be consistent for all b ∈ R n – Since A has rank m then there is a leading one in every row. Hence [ A | b ] cannot have a row [ 0 , 0 , . . . , 0 , 1 ] = ⇒ rank A � < rank ( A | b ) – [ A | b ] has also m rows = ⇒ rank ( A ) � > rank ( A | b ) – Hence, rank ( A ) = rank ( A | b ) Example     1 2 1 1 1 0 − 3 0  → · · · → B = 2 3 0 5 0 1 2 0 rank ( B ) = 3    3 5 1 4 0 0 0 1 Any system B x = d in 4 unknowns and 3 equalities with d ∈ R 3 is consistent. Since rank ( A ) is smaller than the number of variables, then there is a non-leading variable. Hence infinitely many solutions! 11

Numerical Issues in Python Rank Range Example     1 3 − 2 0 0 0 1 3 0 4 0 − 28 0 0 1 2 3 1 0 0 1 2 0 − 14     [ A | b ] =  → · · · →     0 0 0 0 1 5 0 0 0 0 1 5    0 0 0 0 0 0 0 0 0 0 0 0 rank ([ A | b ]) = 3 < 5 = n x 1 + 3 x 2 + 4 x 4 = − 28 x 3 + 2 x 4 = − 14 x 5 = 5 x 1 , x 3 , x 5 are leading variables; x 2 , x 4 are non-leading variables (set them to s , t ∈ R )         x 1 = − 28 − 3 s − 4 t − 28 − 3 − 4 x 1 x 2 = s 0 1 0 x 2                 x 3 = − 14 − 2 t x = = − 14 + 0 s + − 2 x 3 t                 x 4 = t 0 0 1 x 4         x 5 = 5 5 0 0 x 5 12

Numerical Issues in Python Rank Summary Range Let A x = b be a general linear system in n variables and m equations: • If rank ( A ) = r < m and rank ( A | b ) = r + 1 then the system is inconsistent. (the row echelon form of the augmented matrix has a row [ 0 0 . . . 0 1 ] ) • If rank ( A ) = r = rank ( A | b ) then the system is consistent and there are n − r free variables; if r < n there are infinitely many solutions, if r = n there are no free variables and the solution is unique Let A x = 0 be an homogeneous system in n variables and m equations, rank ( A ) = r (always consistent): • if r < n there are infinitely many solutions, if r = n there are no free variables and the solution is unique, x = 0 . 13

Numerical Issues in Python Rank General solutions in vector notation Range Example         − 28 − 3 − 4 x 1 0 1 0 x 2                 x = = − 14 + 0 s + − 2 t , ∀ s , t ∈ R x 2                 0 0 1 x 4         5 0 0 x 5 For A x = b : x = p + α 1 v 1 + α 2 v 2 + · · · + α n − r v n − r , ∀ α i ∈ R , i = 1 , . . . , n − r Note: – if α i = 0 , ∀ i = 1 , . . . , n − r then A p = b , ie, p is a particular solution – if α 1 = 1 and α i = 0 , ∀ i = 2 , . . . , n − r then A p = b A ( p + v 1 ) = b − → A p + A v 1 = b − − − → A v 1 = 0 14

Numerical Issues in Python Rank Range Thus (recall that x = p + z , z ∈ N ( A ) ): • If A is an m × n matrix of rank r , the general solutions of A x = b is the sum of: • a particular solution p of the system A x = b and • a linear combination α 1 v 1 + α 2 v 2 + · · · + α n − r v n − r of solutions v 1 , v 2 , · · · , v n − r of the homogeneous system A x = 0 • If A has rank n , then A x = 0 only has the solution x = 0 and so A x = b has a unique solution: p 15

Numerical Issues in Python Rank Outline Range 1. Numerical Issues in Python 2. Rank 3. Range 16

Numerical Issues in Python Rank Range Range Definition (Range of a matrix) Let A be an m × n matrix, the range of A , denoted by R ( A ) , is the subset of R m given by R ( A ) = { A x | x ∈ R n } That is, the range is the set of all vectors y ∈ R m of the form y = A x for some x ∈ R n , or all y ∈ R m for which the system A x = y is consistent. 17

Numerical Issues in Python Rank Range Recall, if x = ( α 1 , α 2 , . . . , α n ) T is any vector in R n and     a 11 a 12 · · · a 1 n a 1 i a 21 a 22 · · · a 2 n a 2 i     a i = i = 1 , . . . , n . A = . . . .  ...     , . . . .     . . . .    a m 1 a m 2 · · · a mn a mi � a 1 a 2 · · · a n � Then A = and A x = α 1 a 1 + α 2 a 2 + . . . + α n a n that is, vector A x in R n as a linear combination of the column vectors of A Hence R ( A ) is the set of all linear combinations of the columns of A . � the range is also called the column space of A : R ( A ) = { α 1 a 1 + α 2 a 2 + . . . + α n a n | α 1 , α 2 , . . . , α n ∈ R } Thus, A x = b is consistent iff b is in the range of A , ie, a linear combination of the columns of A 18