# 11.2: The Basics of Matrix Algebra

- Page ID
- 7254

A matrix is a rectangular array of numbers with rows and columns. As noted, operations performed on matrices are performed on all elements of a matrix simultaneously. In this section, we provide the basic understanding of matrix algebra that is necessary to make sense of the expression of multiple regression in matrix form.

## 11.2.1 Matrix Basics

The individual numbers in a matrix are referred to as “elements”. The elements of a matrix can be identified by their location in a row and column, denoted as Ar,cAr,c. In the following example, mm will refer to the matrix row and nn will refer to the column.

Am,n=⎡⎢ ⎢ ⎢ ⎢ ⎢⎣a1,1a1,2⋯a1,na2,1a2,2⋯a2,n⋮⋮⋱⋮am,1am,2⋯am,n⎤⎥ ⎥ ⎥ ⎥ ⎥⎦Am,n=[a1,1a1,2⋯a1,na2,1a2,2⋯a2,n⋮⋮⋱⋮am,1am,2⋯am,n]

Therefore, in the following matrix;

A=[1058−1210]A=[1058−1210]

element a2,3=0a2,3=0 and a1,2=5a1,2=5.

## 11.2.2 Vectors

A vector is a matrix with single column or row. Here are some examples:

A=⎡⎢ ⎢ ⎢⎣6−1811⎤⎥ ⎥ ⎥⎦A=[6−1811]

or

A=[1287]A=[1287]

## 11.2.3 Matrix Operations

There are several “operations” that can be performed with and on matrices. Most of the these can be computed with `R`

, so we will use `R`

examples as we go along. As always, you will understand the operations better if you work the problems in `R`

as we go. There is no need to load a data set this time – we will enter all the data we need in the examples.

## 11.2.4 Transpose

Transposing, or taking the “prime” of a matrix, switches the rows and columns.^{21} The matrix

A=[1058−1210]A=[1058−1210]

Once transposed is:

A′=⎡⎢⎣10−125180⎤⎥⎦A′=[10−125180]

Note that the operation “hinges” on the element in the upper right-hand corner of AA, A1,1A1,1, so the first column of AA becomes the first row on A′A′. To transpose a matrix in `R`

, create a matrix object then simply use the `t`

command.

`A <- `**matrix**(**c**(10,-12,5,1,8,0),2,3)
A

```
## [,1] [,2] [,3]
## [1,] 10 5 8
## [2,] -12 1 0
```

**t**(A)

```
## [,1] [,2]
## [1,] 10 -12
## [2,] 5 1
## [3,] 8 0
```

## 11.2.5 Adding Matrices

To add matrices together, they must have the same * dimensions*, meaning that the matrices must have the same number of rows and columns. Then, you simply add each element to its counterpart by row and column. For example:

A=[4−320]+B=[814−5]=A+B=[4+8−3+12+40+(−5)]=[12−26−5]A=[4−320]+B=[814−5]=A+B=[4+8−3+12+40+(−5)]=[12−26−5]

To add matrices together in `R`

, simply create two matrix objects and add them together.

`A <- `**matrix**(**c**(4,2,-3,0),2,2)
A

```
## [,1] [,2]
## [1,] 4 -3
## [2,] 2 0
```

`B <- `**matrix**(**c**(8,4,1,-5),2,2)
B

```
## [,1] [,2]
## [1,] 8 1
## [2,] 4 -5
```

`A + B`

```
## [,1] [,2]
## [1,] 12 -2
## [2,] 6 -5
```

See – how easy is that? No need to be afraid of a little matrix algebra!

## 11.2.6 Multiplication of Matrices

To multiply matrices they must be **conformable**, which means the number of * columns* in the first matrix must match the number of

*in the second matrix.*

*rows*ArXq∗BqXc=CrXcArXq∗BqXc=CrXc

Then, multiply column elements by the row elements, as shown here:

A=⎡⎢⎣25106−2⎤⎥⎦∗B=[421572]=AXB=⎡⎢⎣(2X4)+(5X5)(2X2)+(5X7)(2X1)+(5X2)(1X4)+(0X5)(1X2)+(0X7)(1X1)+(0X2)(6X4)+(−2X5)(6X2)+(−2X7)(6X1)+(−2X2)⎤⎥⎦=⎡⎢⎣33391242114−22⎤⎥⎦A=[25106−2]∗B=[421572]=AXB=[(2X4)+(5X5)(2X2)+(5X7)(2X1)+(5X2)(1X4)+(0X5)(1X2)+(0X7)(1X1)+(0X2)(6X4)+(−2X5)(6X2)+(−2X7)(6X1)+(−2X2)]=[33391242114−22]

To multiply matrices in `R`

, create two matrix objects and multiply them using the `\%*\%`

command.

`A <- `**matrix**(**c**(2,1,6,5,0,-2),3,2)
A

```
## [,1] [,2]
## [1,] 2 5
## [2,] 1 0
## [3,] 6 -2
```

`B <- `**matrix**(**c**(4,5,2,7,1,2),2,3)
B

```
## [,1] [,2] [,3]
## [1,] 4 2 1
## [2,] 5 7 2
```

`A %*% B`

```
## [,1] [,2] [,3]
## [1,] 33 39 12
## [2,] 4 2 1
## [3,] 14 -2 2
```

## 11.2.7 Identity Matrices

The identity matrix is a square matrix with 1’s on the diagonal and 0’s elsewhere. For a 4 x 4 matrix, it looks like this:

I=⎡⎢ ⎢ ⎢⎣1000010000100001⎤⎥ ⎥ ⎥⎦I=[1000010000100001]

It acts like a 1 in algebra; a matrix (AA) times the identity matrix (II) is AA. This can be demonstrated in `R`

.

`A <- `**matrix**(**c**(5,3,2,4),2,2)
A

```
## [,1] [,2]
## [1,] 5 2
## [2,] 3 4
```

`I <- `**matrix**(**c**(1,0,0,1),2,2)
I

```
## [,1] [,2]
## [1,] 1 0
## [2,] 0 1
```

`A %*% I`

```
## [,1] [,2]
## [1,] 5 2
## [2,] 3 4
```

Note that, if you want to square a column matrix (that is, multiply it by itself), you can simply take the transpose of the column (thereby making it a row matrix) and multiply them. The square of column matrix AA is A′AA′A.

## 11.2.8 Matrix Inversion

The matrix inversion operation is a bit like dividing any number by itself in algebra. An inverse of the AA matrix is denoted A−1A−1. Any matrix multiplied by its inverse is equal to the identity matrix:

AA−1=A−1A=IAA−1=A−1A=I

For example,

A=[1−1−1−1]and A−1=[0.5−0.5−0.50.5]therefore A∗A−1=[1001]A=[1−1−1−1]and A−1=[0.5−0.5−0.50.5]therefore A∗A−1=[1001]

However, matrix inversion is only applicable to a square (i.e., number of rows equals number of columns) matrix; only a square matrix can have an inverse.

### Finding the Inverse of a Matrix

To find the inverse of a matrix, the values that will produce the identity matrix, create a second matrix of variables and solve for II.

A=[3124]X[abcd]=[3a+b3c+d2a+4b2c+4d]=[1001]A=[3124]X[abcd]=[3a+b3c+d2a+4b2c+4d]=[1001]

Set 3a+b=13a+b=1 and 2a+4b=02a+4b=0 and solve for aa and bb. In this case a=25a=25 and b=−15b=−15. Likewise, set 3c+d=03c+d=0 and 2c+4d=12c+4d=1; solving for cc and dd produces c=−110c=−110 and d=310d=310. Therefore,

A−1=[25−110−15310]A−1=[25−110−15310]

Finding the inverse matrix can also be done in `R`

using the `solve`

command.

`A <- `**matrix**(**c**(3,2,1,4),2,2)
A

```
## [,1] [,2]
## [1,] 3 1
## [2,] 2 4
```

`A.inverse <- `**solve**(A)
A.inverse

```
## [,1] [,2]
## [1,] 0.4 -0.1
## [2,] -0.2 0.3
```

`A %*% A.inverse`

```
## [,1] [,2]
## [1,] 1 0
## [2,] 0 1
```

OK – now we have all the pieces we need to apply matrix algebra to multiple regression.