Arrays: computing with many numbers Some perspective We have so far - PowerPoint PPT Presentation

Arrays: computing with many numbers

Some perspective — We have so far (mostly) looked at what we can do with single numbers (and functions that return single numbers). — Things can get much more interesting once we allow not just one, but many numbers together. — It is natural to view an array of numbers as one object with its own rules. — The simplest such set of rules is that of a vector.

Vectors A vector is an element of a Vector Space 8 9 x 1 > > > > x 2 > > < = + -vector: x 2 · · · x n ] T x = = [ x 1 . . . > > > > > > x n : ; k d Vector space ! : A vector space is a set " of vectors and a field ℱ of scalars with two operations: se 1) addition: $ + & ∈ " , and $, & ∈ " 2) multiplication : α * $ ∈ " , and $ ∈ " , α ∈ ℱ

Vector Space The addition and multiplication operations must satisfy: (for , , - ∈ ℱ and $ , & ∈ " ) Associativity: $ + & + . = $ + & + . Commutativity: $ + & = & + $ Additive identity: & + 0 = & Additive inverse: & + −& = 0 Associativity wrt scalar multiplication: , * - * & = , * - * & Distributive wrt scalar addition: , + - * & = , * & + - * & Distributive wrt vector addition: , * $ + & = , * $ + , * & Scalar multiplication identity: 1 * $ = $

Linear Functions Function: 00 set & set $ (“output data”) (“input data”) The function ! takes vectors " ∈ $ and transforms into vectors % ∈ & Ida A function f is a linear function if (1) f ( u + v ) = f ( u ) + f ( v ) → a (2) f ( a u ) = a f ( u ) for any scalar a

⇒ Linear functions? - lute Hutu ) - 3 4 = |"| Uto " , 3: ℛ → ℛ -1¥ f- (a) = # f Co2 - f- Co ) f- ( Utv ) f- flu ) t 3 4 = 5 4 + 6 , 3: ℛ → ℛ , 5, 6 ∈ ℛ and 5, 6 ≠ 0 flu ) = a ut b } au ta v t 2b = a Vtb = a cut v ) t 2b flu ) = a ( ut Gtb 2¥ Hutu ) =

Matrices D ## D #$ … D #% D $# D $$ … D $% • ;×+ -matrix @ = ⋮ ⋮ ⋱ ⋮ D &# D &$ … D &% • Linear functions 3(>) can be represented by a Matrix-Vector multiplication. • • Think of a matrix @ as a linear function that takes vectors > and transforms them into vectors A - -0 A = 3(>) → A = @ > a - • Hence we have: mm } A @ B + C = @ B + @ C - no @ , B = , @ B →

MX N Matrix-Vector multiplication l Recall summation notation for matrix-vector multiplication % = ( " • txt meh % ) ! = ∑ "#$ + !" , " - = 1,2, … , 2 • You can think about matrix-vector multiplication as: ← ① € = Linear combination of % = , $ 3 : , 1 + , & 3 : , 2 + ⋯ + , % 3[: , 8] column vectors of A = = = 3 1, : : " Dot product of " with % = ⋮ rows of A 3 2, : : "

Matrices operating on data rs M Data set: " Data set: % I Rotation A = H > or A = @ >

Example: Shear operator Matrix-vector multiplication for each vector (point representation " " " :* :i÷ . " , in 2D): ! " (Ia this ) . - Miao a :O , to ! ! , - ¥ : : H¥¥¥ *

Matrices as operators • Data : grid of 2D points • Transform the data using matrix multiply What can matrices do? 1. Shear 2. Rotate 3. Scale 4. Reflect 5. Can they translate?

Rotation operator I # 4 # I $ = cos(M) −sin(M) 4 $ sin(M) cos(M) ' = )/6

Scale operator I # 4 # I $ = 5 0 4 $ 0 6 3/2 0 0 3/2 3/2 0 0 1 3/2 0 0 2/3

Reflection operator I # 4 # I $ = −5 0 4 $ 0 −6 −1 0 −1 0 0 −1 0 1 Reflect about y-axis Reflect about x and y-axis

Translation (shift) I # 4 # + 5 I $ = 1 0 4 $ 6 0 1 ) = 0.6; . = 1.1

Matrices operating on data Data set: ( Data set: < Rotation

Norms O , . I ←

Demo “Vector Norms” Example of Norms O O x,ltlxzlt---.tlxn1p=2 p -4 I : N x ft XE t r l X n f i - - H X Hoo = mfex I Xi I p =D :

t÷÷i÷ ÷÷÷9÷÷ Unit Ball: Set of vectors > with norm > = 1

Norms and Errors # = =

Absolute and Relative Errors ① Her Hp , = Health Xtrue = ( 40.114 , -88.224 ) llxtruellp -2 Xmea = ( 40 , -88 ) 0.2513 = -§a I 40.1142+88.2242 = ( 0.114 , -0.224 ) 22593 × 10-3 0.2513 Her - zedo.1142to.22.TT Heallp . =

Matrix Norms

⇐ I f¥÷÷÷i÷÷÷÷÷÷÷÷ : : : ÷÷÷÷÷i÷÷÷ : vena :

Matrix Norms

Matrix Norms

Induced Matrix Norms ± : % @ # = max S D -, Maximum absolute column sum of the matrix ( , - mash -.# % @ / = max S D -, Maximum absolute row sum of the matrix ( - ±nE¥÷"÷÷ :* = ,.# Kimm "ax @ $ = max T 0 0 = ' are the singular value of the matrix ( -

Properties of Matrix Norms

Examples Determine the norm of the following matrices: 2) II. ① 1) Hoo = 7 H → 1¥41 ⇒ it - 6 up .

{ fight .tt#T3 III. " " = { 10 , 5,30 } HAH → 30

- T = [ 100 , 0.5 ] 13 , 100 HA ftp.z-m.at Ti =

¥77 : : : : : :D ' " a- " m ' T → ' Ts T = [ Too ②

6 Notation and special matrices � Square matrix: / = 0 • ⇢ 1 i = j δ ij = 0 i 6 = j A ij = 0 • Zero matrix: ⇢ [ I ] = [ δ ij ] • Identity matrix [ A ] = [ A ] T • Symmetric matrix: A ij = A ji ⇢ • Permutation matrix: ) 8 0 0 1 . ) = 1 0 0 • Permutation of the identity matrix 8 . 0 1 0 • Permutes (swaps) rows Diagonal matrix: 1 !" = 0, ∀4, 5 | 4 ≠ 5 • • Triangular matrix: Upper triangular: = !" = := !" , 4 ≤ 5 Lower triangular: 9 !" = :9 !" , 4 ≥ 5 0, 4 > 5 0, 4 < 5

More about matrices Rank: the rank of a matrix @ is the dimension of the vector space generated • by its columns, which is equivalent to the number of linearly independent columns of the matrix. - - Suppose @ has shape /×0: • C)0D @ ≤ min(/, 0) • Matrix @ is full rank : C)0D @ = min(/, 0) . Otherwise, matrix @ is • rank deficient . Singular matrix: a square matrix @ is invertible if there exists a square matrix • J such that @K = K@ = L . If the matrix is not invertible, it is called singular.