20/11/2019 20/11/2019 2 Outline Introduction to Linear Algebra • Algebra • Solving equations • Matrices Leonardo de Knegt • Types of matrices • Matrix operations • Systems of linear equations • Linear regression • State-space models 20/11/2019 3 20/11/2019 4 Algebra x Arythmetics Solving equations • Let a ≠ 0 and b be known real numbers, and x be • Arythmetics an unknown real number. • Greek Arythmos : to count • Computation of numbers • If, for some reason, we know that • Operations: sum, subtraction, multiplication, division � ∙ � � � • Arythmetic expression: 7 + 3 = 3 + 7 we say that we have an equation. • Algebra • Arabic Al-jabr : the reunion of broken parts • Combines unknown quantities and numbers • We can solve the equation • Problem solving using generalizable rules � ∙ � � � • Algebraic equation: a + b = b + a � � � � � � �� b 20/11/2019 5 20/11/2019 6 Solving equations What is a matrix? - Example Farmer Hansen has delivered milk to the dairy last • Consider the system week. 3� � + 6� - + 2� . � 0 /1� � + 5� - + 23� . � 0 • He delivered 10.000kg /2� - + 4� . � 0 • He has received a total of 23.000 DKK as payment. What is the price for 1 kg of milk? (Cheat: � ∙ � � � ) • We write the values of x 1 , x 2 and x 3 in matrix form: x 1 x 2 x 3 � � 10.000 �� 3 6 2 • � � 23.000 ��� • /1 5 23 � � ���� ����� �������� • 0 /2 4 So 10.000 ∙ � � 23.000, therefore Which is a 3x3 matrix ( Notation: #8��9 : #;�����9 ) � � 23.000 ��� � (, )* ��� 10.000 �� �� 1
20/11/2019 20/11/2019 7 20/11/2019 8 Matrices Types of matrices • Another example A matrix � of dimension � : � is called a quadratic (or 3 6 2 2x3 /1 5 23 rectangular) matrix: 3 /1 6 5 3x2 2 23 A matrix � of dimension 1 : � is called a row vector: Symbolically we can express them as � �� � �- � �. � -� � -- � -. A matrix � of dimension � : 1 is called a column vector � �� � �- � -� � -- � .� � .- 20/11/2019 9 20/11/2019 10 Matrix operations Matrix operations Class exercise Addition - Two matrices may be added if they are of equal dimensions (say � : � ) Additive identity Does a set of � : � matrices have a ‘zero’ element 0 so that for any � : � + 0 � � From the axioms of real numbers, it follows that the If yes, what does it look like? commutative law is valid for matrix operation: � + � � � + � 2 /1 15 0 0 0 2 /1 15 5 4 /7 0 0 0 5 4 /7 + = 8 12 23 0 0 0 8 12 23 20/11/2019 11 20/11/2019 12 Matrix operations Matrix operations Multiplication Like for real numbers, every matrix � , has an additive inverse, - Two matrices ( � and � ) may be multiplied if a has the � , such that � + � � 0 , and, for the additive inverse, b � /a same number of columns as b has rows r a x c a r b x c b A 3 x 3 matrix multiplied with a 3 x 2 matrix b 2 /1 15 /2 1 /15 0 0 0 5 4 5 4 /7 /5 /4 7 0 0 0 + = An element in the product is 8 12 23 /8 /12 /23 0 0 0 3 6 calculated as the product of a row and a column 1 2 Resulting in the ‘zero’ or ‘null’ matrix. 21 30 2 3 2 15 24 a 1 2 4 22 26 3 2 1 2
20/11/2019 20/11/2019 13 20/11/2019 14 Matrix operations Matrix operations Class exercise Multiplicative identity Does the set of matrices have a ‘one’ element ? � , so that if ? � is an � : � matrix, then for any � : � matrix � , ? � ∙ � � � Does the commutative law a * b = b * a apply to matrices, as it applies to real numbers? If yes: • What must the values of n necessarily be? - Yes Identical matrices • What are the elements of ? � – what does the matrix look like? - No Identity matrix Does a ‘one’ element ? - exist such that for any matrix � of - Not always Zero matrix given dimension, � ∙ ? - � � - Maaaaybe...? If yes: Same questions as before. 2 /1 15 1 0 0 2 /1 15 Why? 5 4 /7 0 1 0 5 4 /7 * = 8 12 23 0 0 1 8 12 23 20/11/2019 15 20/11/2019 16 Matrix operations Exercise C.1.1 – Appendix C In R: Other operations 2 1 0 • A real number � may be multiplied with a matrix • A = matrix(c(2,-3,1,2,0,-1),2) /3 2 /1 1 2 2 4 2*a = 2 ∗ 3 4 � 6 8 1 1 5 6 10 12 2 /1 • B = matrix(c(1,2,-1,1,-1,0),3) /1 0 • The transpose � @ (often denoted as � A or � B ) of a matrix is • Sum: A+B formed by changing the columns to rows. • Multiplication: A%*%B �′ � 1 3 5 • Transposition: t(A) 2 4 6 • Inverse: solve(A) • Determinant: det(A) Visualize it as a folding around the diagonal. 20/11/2019 17 Determinant Matrix operations • The determinant of a quadratic matrix is a real Multiplicative inverse I number . Does every matrix that are non-zero ( � E 0 ) have a • Calculation of the determinant is rather multiplicative inverse, � , such that � ∙ � � ? ? complicated for large dimensions. Only under certain conditions: • The determinant of a 2 x 2 matrix: • Matrix � is quadratic • Matrix � is non-singular • � is singular if and only if F�G � � 0 where F�G � is the determinant of � . • The determinant of a 3 x 3 matrix: • Many quadratic matrices are singular! 3
20/11/2019 20/11/2019 19 20/11/2019 20 Matrix operations Matrix operations The inverse matrix The inverse matrix A 2x2 matrix a is inverted by � �� � � �� � �- �� 1 � -- /� �- � � -� � -- � �� � �- /� -� � �� � -� � -- Example of a 3x3 matrix Example 1 0 �� 1 � 1 1 3 0 /3 0 /3 2 � � 1 3 1 / 1 2 0 /2 1 /6 /2 1 2 0 3 6 20/11/2019 21 20/11/2019 22 Exercise C.1.2 – Appendix C Why? In R: 2 1 0 • Because they enable us to express very complex relations • A = matrix(c(2,-3,1,2,0,-1),2) in a very compact way. /3 2 /1 • Because they enable us to solve large systems of linear 1 1 equations. 2 /1 • B = matrix(c(1,2,-1,1,-1,0),3) /1 0 • Sum: A+B WARNING • Multiplication: A%*%B NOT POSSIBLE to: • Transposition: t(A) • Inverse: solve(A) Divide by a matrix • • Determinant: det(A) Sum a matrix with a number • 20/11/2019 23 20/11/2019 24 Systems of linear equations Systems of linear equations - Old McDonalds Farm Old McDonalds has a farm… • We have two equations with two unknowns: • On his farm he has some sheep , but he has forgotten how many. Let us denote the number as x 1 . • 1 x 1 + 1 x 2 = 25 • 4 x 1 + 2 x 2 = 70 • On his farm he has some geese , but he has forgotten how • Define the following matrix a and the (column-) vectors x many. Let us denote the number as x 2 . and b • He has no other animals, and the other day he counted the number of heads of his animals. The number was 25 . He knows that sheep and geese have one head each, so he set • We may then express the two equations as one matrix up the following equation: equation: • 1 x 1 + 1 x 2 = 25 • He also counted the number of legs , and it was 70 . He knows that a sheep has 4 legs and a goose has 2 legs, so he set up the following equation: • 4 x 1 + 2 x 2 = 70 4
20/11/2019 20/11/2019 25 20/11/2019 26 Systems of linear equations Linear regression and matrices I - Old McDonalds Farm • In a study of children born in Berkeley 1928-29 Remember: the height and weight of 10 18-year old girls � � � �� � were measured. This solution is valid whether it’s a system of 2 equations or a million (which is not unrealistic). • It is reasonable to assume that the weight Y i depends on the height x i according to the following linear regression model : Y i = β 0 + β 1 x i + ε i where, In R: • a = matrix(c(1,1,4,2),2) b = (matrix(c(25,70),2) β 0 and β 1 are unknown parameters • The ε i are N(0, σ 2 ) • x = (solve(a))%*%b Solution: Solving this system we get � � � � � 10 15 , � - meaning 10 sheep and 15 geese. 20/11/2019 27 20/11/2019 28 Linear regression and matrices II Linear regression and matrices III The least squares estimate of β is Let us define the following matrices Define the vector of predictions as: Then an estimate, s 2 , for the residual variance σ 2 is: Where n = 10 is the number of observations and k = 2 is the number of parameters • • We may then write our model in matrix notation simply as: estimated. Y = x β + ε Yielding 20/11/2019 29 20/11/2019 30 Complex relations Complex relations II • Madsen et al. (2005) performed an on-line monitoring of the water intake of piglets. The water intake Y t at time t was expressed as • Where F , θ t and w t are of dimension 25 x 1, G and W t are of dimension 25 x 25. • The value of θ θ θ θ t is what we try to estimate. • Simple, but … 5
20/11/2019 20/11/2019 31 Matrices and state-space models Let’s do exercise C.1.3. a and b together, as an example. 6
Recommend
More recommend
