Linear algebra NEU 466M Instructor: Professor Ila R. Fiete Spring - PowerPoint PPT Presentation

Linear algebra NEU 466M Instructor: Professor Ila R. Fiete Spring 2016

NotaBon • Matrices: upper-case A, B, U, W • Vector: bold , (usually) lower-case (handwriBng: ) x , y , v , w x → x • Elements of matrix, vector: lower-case a ij , b i , v j , u kl • Scalar numbers: lower-case, no indices a, b, c, γ , α

Vectors and matrices   v 1 v 2     size (m x 1) column vector v = .   . v i ∈ R .   v ∈ R m v m   a 11 a 12 a 1 m · · · a 21 a 22 a 2 m  · · ·  A = size (n x m) matrix   · · · · · · · · · · · ·   A ∈ R n × m a n 1 a n 2 a nm · · ·

What is a vector? geometric view   v 1 v 3 v 2     v = .   . v 2 .   v m v 1 size (m x 1) column vector plotv in matlab

Vector length   v 1 v 2 Length (norm):     v = .   . q .   v 2 1 + v 2 || v || = 2 + · · · v 2 m v m || v ||

Vector-scalar product   α v 1 α v 2     α v = .   . .   α v m geometric view α v v same direcBon, different length

Sum of vectors   v 1 + u 1 v 2 + u 2   v , u ∈ R m v + u =   . .   .   v m + u m geometric view v v + u u u v v u Adding vectors: stacking them end-to-end

Unit vector: any vector of length 1 v m u q X u 2 v 2 m = 1 v 2 1 + v 2 || v || = 2 + · · · v 2 m = 1 = t i i =1 e 3 ˆ e 2 ˆ e 1 ˆ Every point on (m-1) -dimensional sphere of unit radius in m- dim space is a unit vector

Vector, matrix transpose   v 1 v 2   v T = [ v 1 v 2 · · · v m ]   v = .   . .   v m size (m x 1) column vector size (1 x m) row vector     a 11 a n 1 a 11 a 12 a 1 m · · · · · · a 21 a 22 a 2 m a 12 a 2 m · · ·  · · ·    A T = A =     · · · · · · · · · · · · · · · · · · · · ·     a n 1 a n 2 a nm a 1 m a nm · · · · · · size (n x m) matrix size (m x n) matrix

Vector norm as an inner product m v T v = [ v 1 v 2 · · · v m ] X v 2 = || v || 2   v 1 = i v 2   i =1   = .   . .   v m

Inner product (dot product) v , u ∈ R m   u T v = [ u 1 u 2 · · · u m ] X v 1 u i v i = v 2   i   = .   . .   v m u T v = || u |||| v || cos( θ ) Geometric view: projecBon of v on u , Bmes norm of u: u v || u |||| v || cos( θ )

Inner product (dot product) u , v ∈ R 2  �  � 1 v 1 Example: unit vector along x -axis, u = v = 0 v 2 u T v = v 1 v u 1

Inner product (dot product) v , u ∈ R m Example: u ⊥ v u T v = || u |||| v || cos( θ ) = 0 v u

System of equaBons n equaBons in m unknowns (v 1 ,…v m ): a 11 v 1 + · · · + a 1 m v m = b 1 a 21 v 1 + · · · + a 2 m v m = b 2 · · · · · · · · · a n 1 v 1 + · · · + a nm v m = b n

System of equaBons n equaBons in m unknowns (v 1 ,…v m ): a 11 v 1 + · · · + a 1 m v m = b 1 a 21 v 1 + · · · + a 2 m v m = b 2 · · · · · · · · · a n 1 v 1 + · · · + a nm v m = b n       b 1 v 1 a 11 a 12 a 1 m · · · b 2 v 2     a 21 a 22 a 2 m  · · ·  =        .   .  · · · · · · · · · · · ·   . .     . . b n a n 1 a n 2 a nm · · · v m (n x m) (m x 1) (n x 1) A v = b

System of equaBons: when does unique soluBon exist? n equaBons in m unknowns: generically , a unique soluBon exists when same number of constraints (n) as unknowns (m) : Thus, n=m or A is square.       b 1 a 11 a 1 m v 1 · · · b 2 v 2   a 21 a 2 m    · · ·   =         .  .  · · · · · · · · · .   .    . . b m a m 1 a mm · · · v m (m x m) (m x 1) (n x 1) A v = b (m x m) (m x 1) (m x 1) m this is an algebraic view. m = Bme for some geometric insight.

Geometric view: when does a unique soluBon exist? Start with 2-dimensional problem: 2 unknowns, 2 equaBons a 11 x 1 + a 12 x 2 = b 1 equaBon of a line unknowns x 1 , x 2 a 21 x 1 + a 22 x 2 = b 2 a 21 x 1 + a 22 x 2 = b 2 x 2 a 11 x 1 + a 12 x 2 = b 1 soluBon: at intersecBon where both equaBons hold x 1

Geometric view: when does a unique soluBon exist? Start with 2-dimensional problem: 2 unknowns, 2 equaBons a 11 x 1 + a 12 x 2 = b 1 equaBon of a line unknowns x 1 , x 2 a 21 x 1 + a 22 x 2 = b 2 a 21 x 1 + a 22 x 2 = b 2 x 2 a 11 x 1 + a 12 x 2 = b 1 soluBon: at intersecBon where both equaBons hold x 1 Generically two infinite lines in 2D space intersect at a (single) locaBon thus (unique) soluBon exists.

Geometric view: when does a unique soluBon not exist? 1. Offset parallel lines: no soluBon exists x 2 a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 b 2 /a 21 b 1 /a 11 x 1

Algebra: when does a unique soluBon not exist? 1. Offset parallel lines: no soluBon exists x 2 a 11 x 1 + a 12 x 2 = b 1 a 21 x 1 + a 22 x 2 = b 2 b 2 /a 21 b 1 /a 11 x 1 a 21 /a 22 = a 11 /a 12 equal slopes a 11 a 22 = a 12 a 21 a 11 a 22 − a 12 a 21 = 0

Algebra: when does a unique soluBon not exist? 2. Aligned parallel lines: infinitely many soluBons x 2 a 21 x 1 + a 22 x 2 = b 2 a 11 x 1 + a 12 x 2 = b 1 b 1 /a 11 x 1 b 2 /a 21 equal slopes a 11 a 22 − a 12 a 21 = 0 equal intercepts b 1 /a 11 = b 2 /a 21

Algebraic view: existence of unique soluBon in terms of coefficient matrix A  � a 11 a 12 A = a 21 a 22 det( A ) ≡ a 11 a 22 − a 12 a 21 = 0 determinant: 2-dim system of equaBons with square coefficient matrix A has a unique soluBon when: det( A ) 6 = 0 Same condiBon for m -dim system of equaBons with square coefficient matrix.

Linear system: possibiliBes • 1 unique soluBon • No soluBons • Infinitely many soluBons