Chapter II: Background Mathematics Information Retrieval & Data - PowerPoint PPT Presentation

Chapter II: Background Mathematics Information Retrieval & Data Mining Universität des Saarlandes, Saarbrücken Winter Semester 2013/14 II.1- 1

Chapter II: Background Mathematics 1. Linear Algebra Matrices, vectors, and related concepts 2. Probability Theory and Statistical Inference Events, probabilities, random variables, and limit theorems; likehoods and estimators 3. Confidence Intervals, Hypothesis Testing and Regression Confidence intervals, statistical tests, linear regression IR&DM, WS'13/14 22 October 2013 II.1- 2

Chapter II.1: Linear Algebra 1. Matrices and vectors 1.1. Definitions 1.2. Basic algebraic operations 2. Basic concepts 2.1. Orthogonality and linear independence 2.2. Rank, invertibility, and pseudo-inverse 3. Fundamental decompositions 3.1. Eigendecomposition 3.2. Singular value decomposition IR&DM, WS'13/14 22 October 2013 II.1- 3

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: – L p norm (1 ≤ p ≤ ∞ ) i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x • The direction is the angle IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) 1,6 1,2 – L p norm (1 ≤ p ≤ ∞ ) 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • The direction is -0,4 the angle -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) 1,6 (2, –0.8) 1,2 – L p norm (1 ≤ p ≤ ∞ ) 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • The direction is -0,4 the angle -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude 2 (1.2, 0.8) 1,6 (2, –0.8) 1,2 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 -0,4 -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) � 1 / 2 1,6 (2, –0.8) � ∑ n i = 1 x 2 k x x k = k x x k 2 = x x i 1,2 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 -0,4 -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) (1.2 2 + 0.8 2 ) 1/2 = 1.442 � 1 / 2 1,6 (2, –0.8) { � ∑ n i = 1 x 2 k x x k = k x x k 2 = x x i 1,2 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 -0,4 -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) (1.2 2 + 0.8 2 ) 1/2 = 1.442 � 1 / 2 1,6 (2, –0.8) { � ∑ n i = 1 x 2 k x x k = k x x k 2 = x x i 1,2 – L p norm (1 ≤ p ≤ ∞ ) 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 -0,4 -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) (1.2 2 + 0.8 2 ) 1/2 = 1.442 � 1 / 2 1,6 (2, –0.8) { � ∑ n i = 1 x 2 k x x k = k x x k 2 = x x i 1,2 – L p norm (1 ≤ p ≤ ∞ ) 0,8 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • The direction is -0,4 the angle -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

Matrices and vectors • A vector is – a 1D array of numbers – a geometric entity with magnitude and direction • The norm of the vector defines its magnitude – Euclidean ( L 2 ) norm: 2 (1.2, 0.8) (1.2 2 + 0.8 2 ) 1/2 = 1.442 � 1 / 2 1,6 (2, –0.8) { � ∑ n i = 1 x 2 k x x k = k x x k 2 = x x i 1,2 – L p norm (1 ≤ p ≤ ∞ ) 0,8 ∠ 0.5880 rad (33.69°) 0,4 i = 1 | x i | p ) 1 / p x k p = ( ∑ n k x x -1,6 -1,2 -0,8 -0,4 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • The direction is -0,4 the angle -0,8 -1,2 IR&DM, WS'13/14 22 October 2013 II.1- 4

IR&DM, WS'13/14 22 October 2013 II.1- 5

Which of the following are matrices? IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices? A womb IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices?  1 0 0 0 0 0  0 1 0 0 0 0     0 0 1 0 0 0     0 0 0 1 0 0     0 0 0 0 1 0   0 0 0 0 0 1 A rectangular array A womb of numbers IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices?  1 0 0 0 0 0  0 1 0 0 0 0     0 0 1 0 0 0     0 0 0 1 0 0     0 0 0 0 1 0   0 0 0 0 0 1 A rectangular array A womb A graph of numbers IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices?  1 0 0 0 0 0  0 1 0 0 0 0     0 0 1 0 0 0     0 0 0 1 0 0     0 0 0 0 1 0   0 0 0 0 0 1 A rectangular array A womb A graph of numbers 3 x + 2 y + z = 39 2 x + 3 y + z = 34 x + 2 y + 3 z = 26 A system of linear equations IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices?  1 0 0 0 0 0  0 1 0 0 0 0     0 0 1 0 0 0     0 0 0 1 0 0     0 0 0 0 1 0   0 0 0 0 0 1 A rectangular array A womb A graph of numbers f 1 ( x , y , z ) = 3 x + 2 y + z 3 x + 2 y + z = 39 f 2 ( x , y , z ) = 2 x + 3 y + z 2 x + 3 y + z = 34 f 3 ( x , y , z ) = x + 2 y + 3 z x + 2 y + 3 z = 26 f 4 ( x , y , z ) = x A system of A linear mapping linear equations IR&DM, WS'13/14 22 October 2013 II.1- 6

Which of the following are matrices?  1 0 0 0 0 0  0 1 0 0 0 0     0 0 1 0 0 0     0 0 0 1 0 0     0 0 0 0 1 0   0 0 0 0 0 1 A rectangular array A womb A graph of numbers 4 f 1 ( x , y , z ) = 3 x + 2 y + z ● 2 3 x + 2 y + z = 39 ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● f 2 ( x , y , z ) = 2 x + 3 y + z ● ● ● ● ● ● ● ● y 0 ● ● ● 2 x + 3 y + z = 34 ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● f 3 ( x , y , z ) = x + 2 y + 3 z ● ● 2 x + 2 y + 3 z = 26 � ● ● f 4 ( x , y , z ) = x 4 � A system of � 4 � 2 0 2 4 x A linear mapping A set of data points linear equations IR&DM, WS'13/14 22 October 2013 II.1- 6

Vectors in IR&DM • All above meanings of matrices and vectors (and more) are important ways to understand them – Different intuitions provide different insights • In IR&DM, the most important one is the vector space model – A document in a vocabulary of n terms is represented as an n -dimensional vector – A customer transaction in a supermarket selling n items is represented as an n -dimensional vector IR&DM, WS'13/14 22 October 2013 II.1- 7

Vectors in IR&DM • All above meanings of matrices and vectors (and more) are important ways to understand them – Different intuitions provide different insights • In IR&DM, the most important one is the vector space model – A document in a vocabulary of n terms is represented as an n -dimensional vector – A customer transaction in a supermarket selling n items is Information represented as an n -dimensional vector Google Data Trek Star (5, 0, 0, 1, 3, …) IR&DM, WS'13/14 22 October 2013 II.1- 7

Matrices in IR&DM Bread Butter Beer Data Matrix Mining     Anna 1 1 0 Book 1 5 0 3 Bob 1 1 1 Book 2 0 0 7     Charlie 0 1 1 Book 3 4 6 5 Customer transactions Document-term matrix Avatar The Matrix Up Jan Jun Sep     Alice 4 2 Saarbr¨ ucken 1 11 10 Bob 3 2 Helsinki 6 . 5 10 . 9 8 . 7     Charlie 5 3 Cape Town 15 . 7 7 . 8 8 . 7 Incomplete rating matrix Cities and monthly temperatures IR&DM, WS'13/14 22 October 2013 II.1- 8