Notes Slide Set 2 Tools Pietro Coretto pcoretto@unisa.it Econometrics Master in Economics and Finance (MEF) Università degli Studi di Napoli “Federico II” Version: Monday 13 th January, 2020 (h15:38) P. Coretto • MEF Tools 1 / 61 Vector spaces Notes A vector space (linear space) is a collection of objects, called vectors, that can be added together, or multiplied by a scalar (scaling). These operations need to satisfy certain axioms. Many objects in mathematics can be organized in vector spaces, and this has the advantage that we can define concepts like distance , length/size or angles in a general way. The latter is useful to extend the notion of parallelism , direction , magnitude , etc., to a broad class of objects. By now we focus on the Euclidean space R K , x = ( x 1 , x 2 , . . . , x K ) ′ is a K -dimensional vector of R K . P. Coretto • MEF Tools 2 / 61
Distances and metrics Notes A metric space is a space where we can measure distance between its objects. A distance is an abstract function d ( · , · ) that measures of how far apart objects. A distance needs to fulfill axioms (positive-definiteness, symmetry, triangle inequality). The Euclidean vector space R K is also a metric space, examples of distances are d 1 ( x , y ) = � K i =1 | x i − y i | �� K i =1 ( x i − y i ) 2 d 2 ( x , y ) = d ∞ ( x , y ) = max {| x i − y i | , i = 1 , 2 , . . . , K } i =1 | x i − y i | p � 1 p , where p ≥ 1 �� K d p ( x , y ) = P. Coretto • MEF Tools 3 / 61 Notes Every distance defines a different way of metricizing the space Take R 2 , and consider the point 0 = (0 , 0) ′ . Fix ε > 0 , and think about “open balls” (neighbourood) of the point 0 x ∈ R 2 : d 1 ( x , 0 ) ≤ ε � � N 1 = x ∈ R 2 : d 2 ( x , 0 ) ≤ ε � � N 2 = x ∈ R 2 : d ∞ ( x , 0 ) ∞ ≤ ε � � N ∞ = P. Coretto • MEF Tools 4 / 61
Notes Take points x = ( x 1 , x 2 ) ′ ∈ [ − 2 , 2] × [ − 2 , 2] , fix ε = 1 , and color in red those points in the open balls d 1 ( · , · ) d 2 ( · , · ) d ∞ ( · , · ) 2 2 2 1 1 1 x 1 0 x 1 0 x 1 0 -1 -1 -1 -2 -2 -2 -2 -1 0 1 2 -2 -1 0 1 2 -2 -1 0 1 2 x 2 x 2 x 2 P. Coretto • MEF Tools 5 / 61 Length, size, magnitude Notes A norm of a vector it’s a non negative function �·� that measures the “ length/size/magnitude ” of a vector. The norm needs to fulfill a set of axioms. Vector space with a norm, are called normed vector space . Various norms for the usual Euclidean space ℓ 1 -norm: � x � 1 = � K i =1 | x i | �� K i =1 x 2 ℓ 2 -norm: � x � 2 = i ℓ ∞ -norm: � x � ∞ = max {| x i | , i = 1 , 2 , . . . , K } i =1 | x i | p � 1 p , where p ≥ 1 �� K ℓ p -norm: � x � p = P. Coretto • MEF Tools 6 / 61
There is an important connection between metric space and normed vector space Notes For given norm �·� we can define a distance: d ( x , y ) = � x − y � In the Euclidean R K space d 1 ( x , y ) = � x − y � 1 = � K i =1 | x i − y i | �� K i =1 ( x i − y i ) 2 d 2 ( x , y ) = � x − y � 2 = d ∞ ( x , y ) = � x − y � ∞ = max {| x i − y i | , i = 1 , 2 , . . . , K } i =1 | x i − y i | p � 1 p , p ≥ 1 �� K d p ( x , y ) = � x − y � p = Also note that this clarifies the meaning of norm as measure of length/size/magnitude . Let 0 be the origin , a fixed reference point of the vector space, then � x � = � x − 0 � = d ( x , 0 ) P. Coretto • MEF Tools 7 / 61 Angles Notes An inner product it’s another function �· , ·� useful for measuring angles between vectors. It also needs to fulfill axioms. The inner product (aka dot product) defined on the Euclidean space R K is K � � x , y � = x · y = x ′ y = x i y i i =1 If we have an inner product for a vector space, we also have norm. Take the Eucliden space: � K � � � � x 2 � x � 2 = � x , x � = � i i =1 Norm + inner product define angles: � x , y � = cos( θ ) � x � 2 � y � 2 where θ is the angle between x and y . P. Coretto • MEF Tools 8 / 61
Notes The Cauchy–Schwarz Inequality is a direct consequence of this equation, in fact: � x , y � cos( θ ) = ∈ [ − 1 , 1] � x � 2 � y � 2 Orthogonality: x and y are orthogonal/perpendicular when θ = 90 ◦ , 270 ◦ , that is cos( θ ) = 0 . x and y are orthogonal if and only if x ′ y = 0 Collinearity. x and y are collinear if they are either along same line or are parallel to each other. I this case θ = 0 ◦ , 180 ◦ which means cos( θ ) = 1 or cos( θ ) = − 1 . P. Coretto • MEF Tools 9 / 61 Linear dependence Notes The vectors in a subset X = { x 1 , x 2 , . . . , x k } of a vector space are linearly dependent (LD) if there exist scalars { a 1 , a 2 , . . . , a k } , not all of them null, such that a 1 x 1 + a 2 x 2 + . . . , a k x k = 0 therefore, assuming (wlog) a 1 � = 0 , x 1 = − a 2 x 2 + − a 3 x 3 + , . . . , + − a k x k a 1 a 1 a 1 The set of vectors in X are linearly independent (LI) if the equation a 1 x 1 + a 2 x 2 + . . . , a k x k = 0 can only hold for a 1 = a 2 = , . . . , = a k = 0 A set S is LI if it is not redundant , in the sense that you cannot express any vector of S as a linear combination of the other elements of S . P. Coretto • MEF Tools 10 / 61
Notes A subset S spans a vector space V if you can build any element of V as a linear combinations of vectors in S . A subset S of a vector space V , is a basis if it is LI and it spans V . Example. Take the units vectors in R 2 , that is � � � � 1 0 e x = , e y = . 0 1 Now it is easy to see that S = { e x , e y } is basis for R 2 , that means that any point p ∈ R 2 can be written as a linear combination of elements of S . For example � � a p = = a e x + b e y b for any two real numbers ( a, b ) P. Coretto • MEF Tools 11 / 61 Orthogonality Notes Assume X = { x 1 , x 2 , . . . , x p } is a set of non-zero vectors that are orthogonal (pairwise), then 1 vectors of X are LI 2 If p = K , X is a basis for R K , in particular an orthogonal basis In the example before S = { e x , e y } is an orthogonal basis for R 2 Having an orthogonal basis for a vector space V is terribly cool! Why? It means that we can compress V in a convenient way. Namely, each x ∈ V can be written as a linear combination of the elements of the orthogonal basis each x ∈ V can be separated in independent contributions P. Coretto • MEF Tools 12 / 61
This is a key concept for the rest of the course. Notes Let’s consider (wlog) R 2 , and take to basis S t = { t 1 , t 2 } , and S u = { u 1 , u 2 } where only S u is an orthogonal basis . Now consider any point x ∈ R 2 . Since these are basis then it must be true that there will be scalars ( a, b ) and ( c, d ) such that x = a t 1 + b t 2 = c u 1 + d u 2 Question: given x , can we always determine its components ( a, b ) or ( c, d ) ? Do these components uniquely relate to x ? P. Coretto • MEF Tools 13 / 61 For the orthogonal basis Notes � x , u 1 � = � c u 1 + d u 2 , u 1 � = c � u 1 , u 1 � + d � u 2 , u 1 � = c � u 1 , u 1 � � x , u 2 � = � c u 1 + d u 2 , u 2 � = c � u 1 , u 2 � + d � u 2 , u 2 � = d � u 2 , u 2 � Therefore, given x , we can always find c = � x , u 1 � d = � x , u 2 � and � u 1 , u 1 � � u 2 , u 2 � Of course this doesn’t happen for the non-orthogonal basis S t . This example extends to more general spaces, not just R K , ant it teaches us the beauty of the concept: orthogonality = possibility to represent each member of the space with a linear decomposition , where its individual components can be uniquely identified P. Coretto • MEF Tools 14 / 61
Beauty of vector spaces with an inner product Notes If a vector space is endowed with a norm, then it has a distance But recall that if a vector space has an inner product, we can use it define a norm... and a distance as a consequence. Therefore, having an inner product means that we can measure angles, size, distance, and define orthogonality. Whatever complicated the vector space is, we know that if it has an orthogonal basis, its very simple to map all its elements into simpler individual components! P. Coretto • MEF Tools 15 / 61 Matrix algebra Notes Algebra: is a set of objects that can be combined with one or more operations based on rules that make the results well defined. Matrix sum, matrix product, along with the determinant and the inverse allow to construct an algebraic system where one can mimic the fact that with real numbers: there exists unitary element such that 1 × x = x × 1 = x for any x ∈ R one can construct the reciprocal x ∗ is such that x × x ∗ = x ∗ × x = 1 , in R you can set x ∗ = 1 /x for any x � = 0 All the burden of certain matrix algebra calculations are essentially needed to achieve these kind of things. P. Coretto • MEF Tools 16 / 61
Recommend
More recommend
