Skip to main content
Statistics LibreTexts

11.2: The Basics of Matrix Algebra

  • Page ID
    7254
  • \( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

    A matrix is a rectangular array of numbers with rows and columns. As noted, operations performed on matrices are performed on all elements of a matrix simultaneously. In this section, we provide the basic understanding of matrix algebra that is necessary to make sense of the expression of multiple regression in matrix form.

    11.2.1 Matrix Basics

    The individual numbers in a matrix are referred to as “elements”. The elements of a matrix can be identified by their location in a row and column, denoted as Ar,cAr,c. In the following example, mm will refer to the matrix row and nn will refer to the column.

    Am,n=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣a1,1a1,2⋯a1,na2,1a2,2⋯a2,n⋮⋮⋱⋮am,1am,2⋯am,n⎤⎥ ⎥ ⎥ ⎥ ⎥⎦Am,n=[a1,1a1,2⋯a1,na2,1a2,2⋯a2,n⋮⋮⋱⋮am,1am,2⋯am,n]

    Therefore, in the following matrix;

    A=[1058−1210]A=[1058−1210]

    element a2,3=0a2,3=0 and a1,2=5a1,2=5.

    11.2.2 Vectors

    A vector is a matrix with single column or row. Here are some examples:

    A=⎡⎢ ⎢ ⎢⎣6−1811⎤⎥ ⎥ ⎥⎦A=[6−1811]

    or

    A=[1287]A=[1287]

    11.2.3 Matrix Operations

    There are several “operations” that can be performed with and on matrices. Most of the these can be computed with R, so we will use R examples as we go along. As always, you will understand the operations better if you work the problems in R as we go. There is no need to load a data set this time – we will enter all the data we need in the examples.

    11.2.4 Transpose

    Transposing, or taking the “prime” of a matrix, switches the rows and columns.21 The matrix

    A=[1058−1210]A=[1058−1210]

    Once transposed is:

    A′=⎡⎢⎣10−125180⎤⎥⎦A′=[10−125180]

    Note that the operation “hinges” on the element in the upper right-hand corner of AA, A1,1A1,1, so the first column of AA becomes the first row on A′A′. To transpose a matrix in R, create a matrix object then simply use the t command.

    A <- matrix(c(10,-12,5,1,8,0),2,3)
    A
    ##      [,1] [,2] [,3]
    ## [1,]   10    5    8
    ## [2,]  -12    1    0
    t(A)
    ##      [,1] [,2]
    ## [1,]   10  -12
    ## [2,]    5    1
    ## [3,]    8    0

    11.2.5 Adding Matrices

    To add matrices together, they must have the same dimensions, meaning that the matrices must have the same number of rows and columns. Then, you simply add each element to its counterpart by row and column. For example:

    A=[4−320]+B=[814−5]=A+B=[4+8−3+12+40+(−5)]=[12−26−5]A=[4−320]+B=[814−5]=A+B=[4+8−3+12+40+(−5)]=[12−26−5]

    To add matrices together in R, simply create two matrix objects and add them together.

    A <- matrix(c(4,2,-3,0),2,2)
    A
    ##      [,1] [,2]
    ## [1,]    4   -3
    ## [2,]    2    0
    B <- matrix(c(8,4,1,-5),2,2)
    B
    ##      [,1] [,2]
    ## [1,]    8    1
    ## [2,]    4   -5
    A + B
    ##      [,1] [,2]
    ## [1,]   12   -2
    ## [2,]    6   -5

    See – how easy is that? No need to be afraid of a little matrix algebra!

    11.2.6 Multiplication of Matrices

    To multiply matrices they must be conformable, which means the number of columns in the first matrix must match the number of rows in the second matrix.

    ArXq∗BqXc=CrXcArXq∗BqXc=CrXc

    Then, multiply column elements by the row elements, as shown here:

    A=⎡⎢⎣25106−2⎤⎥⎦∗B=[421572]=AXB=⎡⎢⎣(2X4)+(5X5)(2X2)+(5X7)(2X1)+(5X2)(1X4)+(0X5)(1X2)+(0X7)(1X1)+(0X2)(6X4)+(−2X5)(6X2)+(−2X7)(6X1)+(−2X2)⎤⎥⎦=⎡⎢⎣33391242114−22⎤⎥⎦A=[25106−2]∗B=[421572]=AXB=[(2X4)+(5X5)(2X2)+(5X7)(2X1)+(5X2)(1X4)+(0X5)(1X2)+(0X7)(1X1)+(0X2)(6X4)+(−2X5)(6X2)+(−2X7)(6X1)+(−2X2)]=[33391242114−22]

    To multiply matrices in R, create two matrix objects and multiply them using the \%*\% command.

    A <- matrix(c(2,1,6,5,0,-2),3,2)
    A
    ##      [,1] [,2]
    ## [1,]    2    5
    ## [2,]    1    0
    ## [3,]    6   -2
    B <- matrix(c(4,5,2,7,1,2),2,3)
    B
    ##      [,1] [,2] [,3]
    ## [1,]    4    2    1
    ## [2,]    5    7    2
    A %*% B
    ##      [,1] [,2] [,3]
    ## [1,]   33   39   12
    ## [2,]    4    2    1
    ## [3,]   14   -2    2

    11.2.7 Identity Matrices

    The identity matrix is a square matrix with 1’s on the diagonal and 0’s elsewhere. For a 4 x 4 matrix, it looks like this:

    I=⎡⎢ ⎢ ⎢⎣1000010000100001⎤⎥ ⎥ ⎥⎦I=[1000010000100001]

    It acts like a 1 in algebra; a matrix (AA) times the identity matrix (II) is AA. This can be demonstrated in R.

    A <- matrix(c(5,3,2,4),2,2)
    A
    ##      [,1] [,2]
    ## [1,]    5    2
    ## [2,]    3    4
    I <- matrix(c(1,0,0,1),2,2)
    I
    ##      [,1] [,2]
    ## [1,]    1    0
    ## [2,]    0    1
    A %*% I
    ##      [,1] [,2]
    ## [1,]    5    2
    ## [2,]    3    4

    Note that, if you want to square a column matrix (that is, multiply it by itself), you can simply take the transpose of the column (thereby making it a row matrix) and multiply them. The square of column matrix AA is A′AA′A.

    11.2.8 Matrix Inversion

    The matrix inversion operation is a bit like dividing any number by itself in algebra. An inverse of the AA matrix is denoted A−1A−1. Any matrix multiplied by its inverse is equal to the identity matrix:

    AA−1=A−1A=IAA−1=A−1A=I

    For example,

    A=[1−1−1−1]and A−1=[0.5−0.5−0.50.5]therefore A∗A−1=[1001]A=[1−1−1−1]and A−1=[0.5−0.5−0.50.5]therefore A∗A−1=[1001]

    However, matrix inversion is only applicable to a square (i.e., number of rows equals number of columns) matrix; only a square matrix can have an inverse.

    Finding the Inverse of a Matrix

    To find the inverse of a matrix, the values that will produce the identity matrix, create a second matrix of variables and solve for II.

    A=[3124]X[abcd]=[3a+b3c+d2a+4b2c+4d]=[1001]A=[3124]X[abcd]=[3a+b3c+d2a+4b2c+4d]=[1001]

    Set 3a+b=13a+b=1 and 2a+4b=02a+4b=0 and solve for aa and bb. In this case a=25a=25 and b=−15b=−15. Likewise, set 3c+d=03c+d=0 and 2c+4d=12c+4d=1; solving for cc and dd produces c=−110c=−110 and d=310d=310. Therefore,

    A−1=[25−110−15310]A−1=[25−110−15310]

    Finding the inverse matrix can also be done in R using the solve command.

    A <- matrix(c(3,2,1,4),2,2)
    A
    ##      [,1] [,2]
    ## [1,]    3    1
    ## [2,]    2    4
    A.inverse <- solve(A)
    A.inverse
    ##      [,1] [,2]
    ## [1,]  0.4 -0.1
    ## [2,] -0.2  0.3
    A %*% A.inverse
    ##      [,1] [,2]
    ## [1,]    1    0
    ## [2,]    0    1

    OK – now we have all the pieces we need to apply matrix algebra to multiple regression.


    This page titled 11.2: The Basics of Matrix Algebra is shared under a CC BY 4.0 license and was authored, remixed, and/or curated by Jenkins-Smith et al. (University of Oklahoma Libraries) via source content that was edited to the style and standards of the LibreTexts platform; a detailed edit history is available upon request.