Linear algebra A brush-up course Anders Ringgaard Kristensen Slide 1
Outline Real numbers • Operations • Linear equations Matrices and vectors Systems of linear equations Slide 2
Let us start with something familiar! Real numbers! The real number system consists of 4 parts: • A set R of all real numbers • A relation < on R. If a, b ∈ R, then a < b is either true or false. It is called the order relation. • A function +: R × R → R . The addition operation • A function · : R × R → R . The multiplication operation. A number of axioms apply to real numbers Slide 3
Axioms for real numbers I Associative laws • a + ( b + c ) = ( a + b ) + c • a · ( b · c ) = ( a · b ) · c Commutative laws • a + b = b + a • a · b = b · a Distributive law • a · ( b + c ) = a · b + a · c Slide 4
Axioms for real numbers II Additive identity (”zero” element) • There exist an element in R called 0 so that, for all a , a + 0 = a Additive inverse • For all a there exists a b so that a + b = 0, and b = − a Multiplicative identity (”one” element) • There exists an element in R called 1 so that, for all a , 1 · a = a Multiplicative inverse • For all a ≠ 0 there exists a b so that a · b = 1, and b = a -1 Slide 5
Solving equations Let a ≠ 0 and b be known real numbers, and x be an unknown real number. If, for some reason, we know that a · x = b, we say that we have an equation. We can solve the equation in a couple of stages using the axioms: a · x = b ⇔ a - 1 · a · x = a - 1 · b ⇔ 1 · x = a - 1 · b ⇔ x = a - 1 · b Slide 6
a · x = b ⇔ x = a - 1 · b Example of a trivial equation Farmer Hansen has delivered 10000 kg milk to the dairy last week. He received a total payment of 23000 DKK. From this information, we can find the milk price per kg ( a = 10000, b = 23000, x = milk price): • 10000 · x = 23000 ⇔ • x = 10000 -1 · 23000 = 0.0001 · 23000 = 2.30 So, the milk price is 2.30 DKK/kg Slide 7
What is a matrix? A matrix is a rectangular table of real numbers arranged in columns and rows. The dimension of a matrix is written as n × m , where n is the number of rows, and m is the number of columns. We may refer to a matrix using a single symbol, like a , b , x etc. Some times we use bold face ( a , b , x ) or underline ( a , b , x ) in order to emphasize that we refer to a matrix and not just a real number. Slide 8
Examples of matrices A 2 × 3 matrix: A 4 x 3 matrix: Symbol notation for a 2 × 2 matrix: Slide 9
Special matrices A matrix a of dimension n × n is called a quadratic matrix: A matrix b of dimension 1 × n is called a row vector : A matrix c of dimension n × 1 is called a column vector : Slide 10
Operations: Addition Two matrices a and b may be added, if they are of same dimension (say n × m ): From the axioms of real numbers, it follows directly that the commutative law is also valid for matrix addition: • a + b = b + a Slide 11
Additive identity? Does the set of n × m matrices have a ”zero” element 0 so that for any a , a + 0 = a If yes, what does it look like? Slide 12
Operations: Multiplication Two matrices a and b may be multiplied, if a is of dimension n × m , and b is of dimension m × k The result is a matrix of dimension n × k . Due to the dimension requirements, it is clear that the commutative law is not valid for matrix multiplication. • Even when b · a exists, most often a · b ≠ b · a Slide 13
Vector multiplication A row vector a of dimension 1 × n may be multiplied with a column vector b of dimension n × 1. The product a · b is a 1 × 1 matrix (i,e. a real number), where as the product b · a is a quadratic n × n matrix: Slide 14
Matrix multiplication revisited A 3 × 3 matrix multiplied with a 3 × 2 matrix 5 4 An element in the product is 3 6 calculated as the product of a row and a column 1 2 21 30 2 3 2 15 24 1 2 4 22 26 3 2 1 Slide 15
Multiplicative identity Does the set of matrices have a ”one” element I 1 , so that if I 1 is an n × m matrix, then for any m × k matrix a, I 1 · a = a If yes: • What must the value of n necessarily be? • What are the elements of I 1 – what does the matrix look like? Does there exist a ”one” element I 2 so that for any matrix a of given dimension, a · I 2 = a If yes: Same questions as before Slide 16
Additive inverse It follows directly from the axioms for real numbers, that every matrix a , has an additive inverse, b , so that a + b = 0 , and, for the additive inverse, b = − a Slide 17
Other matrix operations A real number r may be multiplied with a matrix a The transpose a’ of a matrix a is formed by changing columns to rows and vice versa: Slide 18
Other matrix operations: Examples If r = 2, and then: The transpose a’ of a is Slide 19
Multiplicative inverse I Does every matrix a ≠ 0 have a multiplicative inverse, b , so that a · b = I If yes, • What does it look like? Slide 20
Multiplicative inverse II A matrix a only has a multiplicative inverse under certain conditions: • The matrix a is quadratic (i.e. the dimension is n × n ) • The matrix a is non-singular : • A matrix a is singular if and only if det( a ) = 0, where det( a ) is the determinant of a • For a quadratic zero matrix 0, we have det(0) = 0, so 0 is singular (as expected) • Many other quadratic matrices are singular as well � Slide 21
Determinant The determinant of a quadratic matrix is a real number. Calculation of the determinant is rather complicated for large dimensions. The determinant of a 2 × 2 matrix: The determinant of a 3 × 3 matrix: Slide 22
The (multiplicative) inverse matrix If a quadratic matrix a is non-singular, it has an inverse a -1 , and: • a · a -1 = I • a -1 · a = I The inverse is complicated to find for matrices of high dimension. For real big matrices (millions of rows and columns) inversion is a challenge even to modern computers. Inversion of matrices is crucial in many applications in herd management (and animal breeding) Slide 23
Inversion of ”small” matrices I A 2 × 2 matrix a is inverted as Example Slide 24
Inversion of ”small” matrices II A 3 × 3 matrix a is inverted as Example Slide 25
Why do we need matrices? Because they enable us to express very complex relations in a very compact way. Because the algebra and notation are powerful tools in mathematical proofs for correctness of methods and properties. Because they enable us to solve large systems of linear equations. Slide 26
Complex relations I Modeling of drinking patterns of weaned piglets. Slide 27
Complex relations Madsen et al. (2005) performed an on-line monitoring of the water intake of piglets. The water intake Y t at time t was expressed as Where Simple, but … Slide 28
Complex relations II F , θ t and w t are of dimension 25 × 1, G and W t are of dimension 25 × 25. The value of θ t is what we try to estimate. Slide 29
Systems of linear equations A naïve example: Old McDonald has a farm … On his farm he has some sheep, but he has forgotten how many. Let us denote the number as x 1 . On his farm he has some geese, but he has forgotten how many. Let us denote the number as x 2 . He has no other animals, and the other day he counted the number of heads of his animals. The number was 25. He knows that sheep and geese have one head each, so he set up the following equation: • 1 x 1 + 1 x 2 = 25 He also counted the number of legs, and it was 70. He knows that a sheep has 4 legs and a goose has 2 legs, so he set up the following equation: • 4 x 1 + 2 x 2 = 70 Slide 30
Old McDonald’s animals We have two equations • 1 x 1 + 1 x 2 = 25 • 4 x 1 + 2 x 2 = 70 Define the following matrix a and the (column-) vectors x and b We may then express the two equations as one matrix equation: Slide 31
Solving systems of linear equations Having brought the system of linear equations to the elegant form, solution for x is just as straight forward as with an equation of real a · x = b ⇔ numbers: a - 1 · a · x = a - 1 · b ⇔ I · x = a - 1 · b ⇔ x = a - 1 · b This is true no matter whether we have a system of 2 equations like here, or we have a system of a million equations (which is not at all unrealistic). Slide 32
Linear regression and matrices I In a study of children born in Berkeley 1928-29 the height and weight of 10 18-year old girls were measured. It is reasonable to assume that the weight Y i depends on the height x i according to the following linear regression model: • Y i = β 0 + β 1 x i + ε i where, � β 0 and β 1 are unknown parameters • The ε i are N(0, σ 2 ) Slide 33
Linear regression and matrices II Let us define the following matrices: We may then write our model in matrix notation simply as: Y = x β + ε • Slide 34
Linear regression and matrices III The least squares estimate of β is Slide 35
Linear regression and matrices IV Define the vector of predictions as Then an estimate s 2 for σ 2 is • Where n = 10 is the number of observations and k = 2 is the number of parameters estimated. Applying these formulas yields: Slide 36
Recommend
More recommend
