Basic Concepts in Linear Algebra Department of Mathematics Boise - PowerPoint PPT Presentation

Basic Concepts in Linear Algebra Department of Mathematics Boise State University September 14, 2017 Math 365 Linear Algebra Basics September 14, 2017 1 / 39

Numerical Linear Algebra Linear systems of equations occur in almost every area of the applied science, engineering, and mathematics. Hence, numerical linear algebra is one of the pillars of computational mathematics. Math 365 Linear Algebra Basics September 14, 2017 2 / 39

Linear systems of equations A linear system of m equations and n unknowns can be expressed in the following general form: a 11 x 1 + a 12 x 2 + a 13 x 3 + · · · + a 1 n x n = b 1 , a 21 x 1 + a 22 x 2 + a 23 x 3 + · · · + a 2 n x n = b 2 , a 31 x 1 + a 32 x 2 + a 33 x 3 + · · · + a 3 n x n = b 3 , (1) . . . . . ... . . . . . . . . . . a m 1 x 1 + a m 2 x 2 + a m 3 x 3 + · · · + a mn x n = b m . Here a ij are the coefficients of the systems, b i are the right hand sides (RHS), and x j are the unknown values that must be determined. a ij and b i will be given by the problem. Math 365 Linear Algebra Basics September 14, 2017 3 / 39

Linear systems of equations Linear systems can be classified into the following three types: Square linear system : If the number of equations equals the 1 number of unknowns (i.e. m = n ). Overdetermined system : If the number of equations is greater 2 than the number of unknowns (i.e. m > n ). Underdetermined system : If the number equations is less than 3 the number of unknowns (i.e. m < n ). Math 365 Linear Algebra Basics September 14, 2017 4 / 39

Matrices and vectors A convenient notation to describe a linear system of equations is in terms of matrices and vectors. Math 365 Linear Algebra Basics September 14, 2017 5 / 39

Matrices A matrix is just a table of numbers containing m rows and n columns and can be expressed as:   a 11 a 12 a 13 · · · a 1 n a 21 a 22 a 23 · · · a 2 n     a 31 a 32 a 33 · · · a 3 n   A = .   . . . . ...   . . . . . . . .   a m 1 a m 2 a m 3 · · · a mn We typically use capital letters to denote matrices. We write A ∈ R m × n to denote a matrix with m rows and n columns. � � A common shorthand notation for a matrix is A = a ij , where the values for i and j are understood from the problem. Math 365 Linear Algebra Basics September 14, 2017 6 / 39

Matrices A matrix is just a table of numbers containing m rows and n columns and can be expressed as:   a 11 a 12 a 13 · · · a 1 n a 21 a 22 a 23 · · · a 2 n     a 31 a 32 a 33 · · · a 3 n   A = .   . . . . ...   . . . . . . . .   a m 1 a m 2 a m 3 · · · a mn We typically use capital letters to denote matrices. We write A ∈ R m × n to denote a matrix with m rows and n columns. � � A common shorthand notation for a matrix is A = a ij , where the values for i and j are understood from the problem. Math 365 Linear Algebra Basics September 14, 2017 6 / 39

Matrices A matrix is just a table of numbers containing m rows and n columns and can be expressed as:   a 11 a 12 a 13 · · · a 1 n a 21 a 22 a 23 · · · a 2 n     a 31 a 32 a 33 · · · a 3 n   A = .   . . . . ...   . . . . . . . .   a m 1 a m 2 a m 3 · · · a mn We typically use capital letters to denote matrices. We write A ∈ R m × n to denote a matrix with m rows and n columns. � � A common shorthand notation for a matrix is A = a ij , where the values for i and j are understood from the problem. Math 365 Linear Algebra Basics September 14, 2017 6 / 39

Matrices A matrix is just a table of numbers containing m rows and n columns and can be expressed as:   a 11 a 12 a 13 · · · a 1 n a 21 a 22 a 23 · · · a 2 n     a 31 a 32 a 33 · · · a 3 n   A = .   . . . . ...   . . . . . . . .   a m 1 a m 2 a m 3 · · · a mn We typically use capital letters to denote matrices. We write A ∈ R m × n to denote a matrix with m rows and n columns. � � A common shorthand notation for a matrix is A = a ij , where the values for i and j are understood from the problem. Math 365 Linear Algebra Basics September 14, 2017 6 / 39

Vectors If the matrix only has one row or column then it is called a vector . A column vector with n entries can be expressed as   x 1 x 2     x 3   x = .   .   . .   x n A row vector and can be expressed as � � x = x 1 x 2 x 3 · · · x n . We typically use bold lower-case letters to denote vectors. A column vector with n real entries is denoted by x ∈ R n , while a row vector is denoted by x ∈ R 1 × n . Math 365 Linear Algebra Basics September 14, 2017 7 / 39

Vectors If the matrix only has one row or column then it is called a vector . A column vector with n entries can be expressed as   x 1 x 2     x 3   x = .   .   . .   x n A row vector and can be expressed as � � x = x 1 x 2 x 3 · · · x n . We typically use bold lower-case letters to denote vectors. A column vector with n real entries is denoted by x ∈ R n , while a row vector is denoted by x ∈ R 1 × n . Math 365 Linear Algebra Basics September 14, 2017 7 / 39

Vectors If the matrix only has one row or column then it is called a vector . A column vector with n entries can be expressed as   x 1 x 2     x 3   x = .   .   . .   x n A row vector and can be expressed as � � x = x 1 x 2 x 3 · · · x n . We typically use bold lower-case letters to denote vectors. A column vector with n real entries is denoted by x ∈ R n , while a row vector is denoted by x ∈ R 1 × n . Math 365 Linear Algebra Basics September 14, 2017 7 / 39

Vectors If the matrix only has one row or column then it is called a vector . A column vector with n entries can be expressed as   x 1 x 2     x 3   x = .   .   . .   x n A row vector and can be expressed as � � x = x 1 x 2 x 3 · · · x n . We typically use bold lower-case letters to denote vectors. A column vector with n real entries is denoted by x ∈ R n , while a row vector is denoted by x ∈ R 1 × n . Math 365 Linear Algebra Basics September 14, 2017 7 / 39

Vectors If the matrix only has one row or column then it is called a vector . A column vector with n entries can be expressed as   x 1 x 2     x 3   x = .   .   . .   x n A row vector and can be expressed as � � x = x 1 x 2 x 3 · · · x n . We typically use bold lower-case letters to denote vectors. A column vector with n real entries is denoted by x ∈ R n , while a row vector is denoted by x ∈ R 1 × n . Math 365 Linear Algebra Basics September 14, 2017 7 / 39

Matrix & vector operations Math 365 Linear Algebra Basics September 14, 2017 8 / 39

Matrix & vector operations Math 365 Linear Algebra Basics September 14, 2017 8 / 39

Matrix & vector operations: Transpose Let A ∈ R m × n with entries   a 11 a 12 a 13 · · · a 1 n a 21 a 22 a 23 · · · a 2 n     a 31 a 32 a 33 · · · a 3 n   A = ,   . . . . ...  . . . .  . . . .   a m 1 a m 2 a m 3 · · · a mn then the transpose of A switches the columns of A with the rows, i.e.   · · · a 11 a 21 a 31 a m 1 a 12 a 22 a 32 · · · a m 2     A T = a 13 a 23 a 33 · · · a m 3   .   . . . . ...   . . . . . . . .   a 1 n a 2 n a 3 n · · · a mn Note that A T ∈ R n × m and that ( A T ) T = A . Math 365 Linear Algebra Basics September 14, 2017 9 / 39

Matrix & vector operations: Transpose The transpose can also be applied to vectors. In this case if x is a (column) vector then x T is a row vector:   x 1 x 2     x T = � � x 3   if x = then x 1 x 2 x 3 · · · x n .   .   . .   x n Similarly if x is row vector then x T is a column vector. Math 365 Linear Algebra Basics September 14, 2017 10 / 39

Matrix & vector operations: Matrix addition Let A ∈ R m × n and B ∈ R m × n then the sum of A and B is given by � � A + B = a ij + b ij . This is just the sum of the corresponding entries of the elements of A and B . For this sum to make sense A and B must be the same size. Math 365 Linear Algebra Basics September 14, 2017 11 / 39

Matrix & vector operations: scalar multiplication Let α be a real number and A ∈ R m × n then the product of α and A is given by � � α A = α a ij . Note that this is just α times each entry of A . Math 365 Linear Algebra Basics September 14, 2017 12 / 39