# Some basic facts about vectors and matrices

### Addition rule for matrices

If \(c_1, ... , c_k\) are scalers, and \(A_1,...,A_k\) are all \(m \times n\) matrices, then \(B = c_1A_1+c_2A_2+ ... + c_kA_k\) is an \(m \times n\) matrix with \((i, j)\)-th entry of \(B : B(i, j) = c_1A_1(i, j) + c_2A_2(i, j) + ... + c_kA_k(i, j)\), for all \(i = 1, ... , m; j = 1, ..., n\). (Note sometimes we denote the entries of a matrix by \(A_{i_j}\), and sometimes by \(A(i, j)\). But always the first index is for the row and the second index is for the column).

### Transpose of a matrix

If \(A\) is an \(m \times n\) matrix, then \(A^T\) (spelled \(A\)-transpose) is the \(n \times m\) matrix \(B\) whose \((i, j)\)-th entry \(B_{ij} = A_{ji}\) for all \(i = 1, ... , n; j = 1, ... ,m\).

### Inner product of vectors

If \(x\) and \(y\) are two \(m \times 1\) vectors, then the *inner product *(or, *dot product*) between \(x\) and \(y\) is given by : \(\left \langle x, y \right \rangle\) = \(\sum_{i=1}^{m}x_iy_i\). Note that \(\left \langle x, y \right \rangle = \left \langle y, x \right \rangle\).

### Multiplication of matrices

If \(A\) is an \(m \times n\) matrix and \(B\) is an \(n \times p\) matrix then the product \(AB = C\), say, is defined and it is an \(m \times p\) matrix with \((i, j)\)-th entry : \(C_{ij} = \sum_{k=1}^{n}A_{ik}B_{kj}\) for all \(i = 1, ... , m; j = 1, ... ,p\). Notethat for \(m \times 1\) vectors \(x\) and \(y\), \(\left \langle x, y \right \rangle = x^Ty = y^Tx\). In other words, the \((i, j)\)-th entry of \(AB\) is the inner product of \(i\)-th row of \(A\) and \(j\)-th column of \(B\).

### Special matricies

**Square matrix:**A matrix \(A\) is square if it is \(m \times m\) (that is, number of rows = number of columns).**Symmetric matrix**: An \(m \times m\) (square) matrix \(A\) is symmetric if \(A = A^T\). That is, for all \(1 \leq i, j \leq m, A_{ij} = A_{ji}\).**Diagonal matrix**: A \(m \times m\) matrix with all the entries zero except (possibly) the entries on the diagonal (that is the (\(i , i\))-th entry for all the \(i = 1, ... m\)) is called a diagonal matrix.**Identity matrix:**The \(m \times m\) diagonal matrix with all diagonal entries equal to 1 is called the identity matrix and is denoted by \(I\) (or, \(I_m\)). It has the property that for any \(m \times n\) matrix \(A\) and any \(p x m\) matrix \(B\), \(IA = A\) and \(BI = B\).**One vector:**The \(m \times 1\) vector with all entries equal to 1 is usually called the one vector (non-standard term) and is denoted by**1**(or, \(1_m\)).**Ones matrix:**The \(m \times m\) matrix with all entries equal to 1 is denoted by \(J\) (or, \(J_m\)). Note that \(J_m\) = \(1_m1_{m}^{T}\).**Zero vector:**The \(m \times 1\) vector with all entries zero is called the zero vector and is denoted by \(0\) (or, \(0_m\)).

**Multiplication is not commutative:**If \(A\) and \(B\) are both \(m \times m\) matrices then both \(AB\) and \(BA\) are defined and are \(m \times m\) matrices. However, in general \(AB \neq BA\). Notice that \(I_mB = BI_m = B\), where \(I_m\) is the identity matrix.**Linear independence:**The \(m \times 1\) vectors \(x_1,...,x_k\), (\(k\) arbitrary) are said to be*linearly dependent,*if there exist constants \(c_1,..., c_m\),**not all zero**, such that $$c_1x_1+c_2x_2 + ... + c_mx_m = 0$$ If no such sequence of numbers \(c1, ... , c_m\) exists then the vectors \(x_1, ..., x_m\) are said to be*linearly independent.*

**Relationship with dimension:**If \(k > m\) then \(m \times 1\) vectors \(x_1, ..., x_k\) are**always**linearly dependent.**Rank of a matrix:**For an \(m \times n\) matrix \(A\), the**rank**of \(A\), written rank(\(A\)) is the maximal number of linearly independent columns of \(A\) (treating each column as an \(m \times 1\) vector). Also, rank(\(A\))\(\leq\)min\(\{m,n\}\)**Nonsingular matrix:**If an \(m \times m\) matrix \(A\) has*full rank,*that is, rank(\(A\)) = \(m\), (which is equivalent to saying that all the columns of \(A\) are linearly independent), then the matrix \(A\) is called**nonsingular**

**Inverse of a matrix:**If an \(m \times m\) matrix \(A\)*nonsingular,*then it has an*inverse,*

that is a*unique*\(m \times m\) matrix denoted by \(A^{-1}\) that satisfies the relationship : \(A^{-1}A = I_m = AA^{-1}\)

**Inverse of a **\(2 \times 2\) **matrix: **Let a \(2 \times 2\) matrix \(A\) be expressed as \(A = \begin{bmatrix}

a &b \\

c &d

\end{bmatrix}\).

Then \(A\) is nonsingular (and hence has an inverse) *if and only if *\(ad-bc \neq 0\). If this is satisfied then the inverse is $$A^{-1} = \frac{1}{ad-bc}\begin{bmatrix}

d & -b\\

-c & a

\end{bmatrix}$$

2. **Solution of a system of linear equations : **A system of \(m\) linear equations in \(m\) variables \(b_1, ...,, b_m\) can be expressed as $$a_{1_1}b_1 + a_{1_2}b_2 + ... + a_{1_m}b_m = c_1$$ $$a_{2_1}b_1 + a_{2_2}b_2 + ... + a_{2_m}b_m = c_2$$ $$ ... ... ... ... ... = .$$ $$a_{m_1}b_1 + a_{m_2}b_2 + ... + a_{m_m}b_m = c_m $$

Here the *coefficients *\(a_{i_j}\) and the constants \(c_i\) are considered unknown. This system can be expressed in matrix form as \(Ab** = **c\), where \(A\) is the \(m \times m\) matrix with \((i, j)\)-th entry \(a_{i_j}\), and **b **and **c **are \(m \times 1\) vectors with \(i\)-th entries \(b_i\) and \(c_i\), respectively, for \(i = 1, ... , m; j = 1, ..., m\).

If the matrix \(A\) is nonsingular, then a *unique solution *exists for this system of equations and is given by **b = **\(A^{-1}c\). To see this, note that since \(A(A^{-1}) = (AA^{-1})c= I c= c\), it shows that \(A^{-1}c\) is a solution. On the other hand, if \(b = b^*\) is a solution, then it satisfies \(Ab^* = c\). Hence \(b^* = I b^* = (A^{-1}A)b^* = A^{-1}(Ab^*) = A^{-1}c\), which proves uniqueness.

### Contributors

- Debashis Paul
- Cathy Wang