Outline Real numbers • Operations • Linear equations Linear algebra Matrices and vectors A brush-up course Systems of linear equations Anders Ringgaard Kristensen Slide 1 Slide 2 Axioms for real numbers I Let us start with something familiar! Associative laws • a + ( b + c ) = ( a + b ) + c Real numbers! • a · ( b · c ) = ( a · b ) · c The real number system consists of 4 parts: Commutative laws • A set R of all real numbers • a + b = b + a • A relation < on R. If a, b ∈ R, then a < b is either true or • a · b = b · a false. It is called the order relation. Distributive law • A function +: R × R → R . The addition operation • a · ( b + c ) = a · b + a · c • A function · : R × R → R . The multiplication operation. A number of axioms apply to real numbers Slide 3 Slide 4 Axioms for real numbers II Solving equations Let a ≠ 0 and b be known real numbers, Additive identity (”zero” element) • There exist an element in R called 0 so that, for all a , a + 0 = a Additive inverse and x be an unknown real number. • For all a there exists a b so that a + b = 0, and b = − a If, for some reason, we know that a · x = b, Multiplicative identity (”one” element) we say that we have an equation. • There exists an element in R called 1 so that, for all a , 1 · a = a Multiplicative inverse • For all a ≠ 0 there exists a b so that a · b = 1, and b = a -1 We can solve the equation in a couple of stages using the axioms: a · x = b ⇔ a - 1 · a · x = a - 1 · b ⇔ 1 · x = a - 1 · b ⇔ x = a - 1 · b Slide 5 Slide 6 1
a · x = b ⇔ x = a - 1 · b What is a matrix? Example of a trivial equation Farmer Hansen has delivered 10000 kg milk to A matrix is a rectangular table of real numbers the dairy last week. He received a total arranged in columns and rows. The dimension of a matrix is written as n × m , payment of 23000 DKK. From this information, we can find the milk price per kg ( a = 10000, b where n is the number of rows, and m is the = 23000, x = milk price): number of columns. • 10000 · x = 23000 ⇔ We may refer to a matrix using a single symbol, • x = 10000 -1 · 23000 = 0.0001 · 23000 = 2.30 like a , b , x etc. Some times we use bold face ( a , So, the milk price is 2.30 DKK/kg b , x ) or underline ( a , b , x ) in order to emphasize that we refer to a matrix and not just a real number. Slide 7 Slide 8 Examples of matrices Special matrices A 2 × 3 matrix: A matrix a of dimension n × n is called a quadratic matrix: A 4 x 3 matrix: A matrix b of dimension 1 × n is called a row vector : Symbol notation for a 2 × 2 matrix: A matrix c of dimension n × 1 is called a column vector : Slide 9 Slide 10 Operations: Addition Additive identity? Two matrices a and b may be added, if they are of same dimension (say n × m ): Does the set of n × m matrices have a ”zero” element 0 so that for any a , a + 0 = a If yes, what does it look like? From the axioms of real numbers, it follows directly that the commutative law is also valid for matrix addition: • a + b = b + a Slide 11 Slide 12 2
Vector multiplication Operations: Multiplication A row vector a of dimension 1 × n may be multiplied with a column vector b of Two matrices a and b may be multiplied, if a is of dimension n × 1. The product a · b is a 1 × dimension n × m , and b is of dimension m × k 1 matrix (i,e. a real number), where as the product b · a is a quadratic n × n matrix: The result is a matrix of dimension n × k . Due to the dimension requirements, it is clear that the commutative law is not valid for matrix multiplication. • Even when b · a exists, most often a · b ≠ b · a Slide 13 Slide 14 Matrix multiplication revisited Multiplicative identity Does the set of matrices have a ”one” element I 1 , so that if I 1 is an n × m matrix, then for any A 3 × 3 matrix multiplied with a 3 × 2 matrix m × k matrix a, I 1 · a = a If yes: • What must the value of n necessarily be? 5 4 • What are the elements of I 1 – what does the matrix An element in the product is look like? 3 6 calculated as the product of a row Does there exist a ”one” element I 2 so that for any matrix a of given dimension, a · I 2 = a and a column 1 2 21 30 If yes: Same questions as before 2 3 2 15 24 1 2 4 22 26 3 2 1 Slide 15 Slide 16 Additive inverse Other matrix operations It follows directly from the axioms for real numbers, that every matrix a , has an A real number r may be multiplied with a matrix a additive inverse, b , so that a + b = 0 , and, for the additive inverse, b = − a The transpose a’ of a matrix a is formed by changing columns to rows and vice versa: Slide 17 Slide 18 3
Multiplicative inverse I Other matrix operations: Examples Does every matrix a ≠ 0 have a multiplicative inverse, b , so that a · b = I If r = 2, and then: If yes, • What does it look like? The transpose a’ of a is Slide 19 Slide 20 Multiplicative inverse II Determinant The determinant of a quadratic matrix is A matrix a only has a multiplicative inverse a real number. under certain conditions: Calculation of the determinant is rather • The matrix a is quadratic (i.e. the dimension complicated for large dimensions. is n × n ) The determinant of a 2 × 2 matrix: • The matrix a is non-singular : • A matrix a is singular if and only if det( a ) = 0, where det( a ) is the determinant of a The determinant of a 3 × 3 matrix: • For a quadratic zero matrix 0, we have det(0) = 0, so 0 is singular (as expected) • Many other quadratic matrices are singular as well � Slide 21 Slide 22 Inversion of ”small” matrices I The (multiplicative) inverse matrix A 2 × 2 matrix a is inverted as If a quadratic matrix a is non-singular, it has an inverse a -1 , and: • a · a -1 = I • a -1 · a = I The inverse is complicated to find for matrices of high dimension. Example For real big matrices (millions of rows and columns) inversion is a challenge even to modern computers. Inversion of matrices is crucial in many applications in herd management (and animal breeding) Slide 23 Slide 24 4
Inversion of ”small” matrices II Why do we need matrices? A 3 × 3 matrix a is inverted as Because they enable us to express very complex relations in a very compact way. Because the algebra and notation are powerful tools in mathematical proofs for correctness of methods and properties. Because they enable us to solve large systems of linear equations. Example Slide 25 Slide 26 Complex relations Complex relations I Madsen et al. (2005) performed an on-line monitoring of the water intake of Modeling of drinking patterns of weaned piglets. piglets. The water intake Y t at time t was expressed as Where Simple, but … Slide 27 Slide 28 Complex relations II Systems of linear equations A naïve example: Old McDonald has a farm … On his farm he has some sheep, but he has forgotten how many. Let us denote the number as x 1 . On his farm he has some geese, but he has forgotten how many. Let us denote the number as x 2 . He has no other animals, and the other day he counted the number of heads of his animals. The number was 25. He knows that sheep and geese have one head each, so he set up the following equation: • 1 x 1 + 1 x 2 = 25 He also counted the number of legs, and it was 70. He F , θ t and w t are of dimension 25 × 1, G and W t are of dimension knows that a sheep has 4 legs and a goose has 2 legs, so he set up the following equation: 25 × 25. The value of θ t is what we try to estimate. • 4 x 1 + 2 x 2 = 70 Slide 29 Slide 30 5
Old McDonald’s animals Solving systems of linear equations We have two equations • 1 x 1 + 1 x 2 = 25 • 4 x 1 + 2 x 2 = 70 Having brought the system of linear equations to Define the following matrix a and the (column-) the elegant form, solution for x is just as vectors x and b straight forward as with an equation of real a · x = b ⇔ numbers: a - 1 · a · x = a - 1 · b ⇔ I · x = a - 1 · b ⇔ x = a - 1 · b We may then express the two equations as one matrix equation: This is true no matter whether we have a system of 2 equations like here, or we have a system of a million equations (which is not at all unrealistic). Slide 31 Slide 32 Linear regression and matrices II Linear regression and matrices I Let us define the following matrices: In a study of children born in Berkeley 1928-29 the height and weight of 10 18-year old girls were measured. It is reasonable to assume that the weight Y i depends on the height x i according to the following linear regression model: • Y i = β 0 + β 1 x i + ε i where, � β 0 and β 1 are unknown parameters • The ε i are N(0, σ 2 ) We may then write our model in matrix notation simply as: Y = x β + ε • Slide 33 Slide 34 Linear regression and matrices III Linear regression and matrices IV The least squares estimate of β is Define the vector of predictions as Then an estimate s 2 for σ 2 is • Where n = 10 is the number of observations and k = 2 is the number of parameters estimated. Applying these formulas yields: Slide 35 Slide 36 6
Recommend
More recommend