Some basic facts about vectors and matrices

Addition rule for matrices

If \(c_1, ... , c_k\) are scalers, and \(A_1,...,A_k\) are all \(m \times n\) matrices, then \(B = c_1A_1+c_2A_2+ ... + c_kA_k\) is an \(m \times n\) matrix with \((i, j)\)-th entry of \(B : B(i, j) = c_1A_1(i, j) + c_2A_2(i, j) + ... + c_kA_k(i, j)\), for all \(i = 1, ... , m; j = 1, ..., n\). (Note sometimes we denote the entries of a matrix by \(A_{i_j}\), and sometimes by \(A(i, j)\). But always the first index is for the row and the second index is for the column).

Transpose of a matrix

If \(A\) is an \(m \times n\) matrix, then \(A^T\) (spelled \(A\)-transpose) is the \(n \times m\) matrix \(B\) whose \((i, j)\)-th entry \(B_{ij} = A_{ji}\) for all \(i = 1, ... , n; j = 1, ... ,m\).

Inner product of vectors

If \(x\) and \(y\) are two \(m \times 1\) vectors, then the inner product (or, dot product) between \(x\) and \(y\) is given by : \(\left \langle x, y \right \rangle\) = \(\sum_{i=1}^{m}x_iy_i\). Note that \(\left \langle x, y \right \rangle = \left \langle y, x \right \rangle\).

Multiplication of matrices

If \(A\) is an \(m \times n\) matrix and \(B\) is an \(n \times p\) matrix then the product \(AB = C\), say, is defined and it is an \(m \times p\) matrix with \((i, j)\)-th entry : \(C_{ij} = \sum_{k=1}^{n}A_{ik}B_{kj}\) for all \(i = 1, ... , m; j = 1, ... ,p\). Notethat for \(m \times 1\) vectors \(x\) and \(y\), \(\left \langle x, y \right \rangle = x^Ty = y^Tx\). In other words, the \((i, j)\)-th entry of \(AB\) is the inner product of \(i\)-th row of \(A\) and \(j\)-th column of \(B\).

Special matricies

1. Square matrix: A matrix \(A\) is square if it is \(m \times m\) (that is, number of rows = number of columns).
2. Symmetric matrix: An \(m \times m\) (square) matrix \(A\) is symmetric if \(A = A^T\). That is, for all \(1 \leq i, j \leq m, A_{ij} = A_{ji}\).
3. Diagonal matrix: A \(m \times m\) matrix with all the entries zero except (possibly) the entries on the diagonal (that is the (\(i , i\))-th entry for all the \(i = 1, ... m\)) is called a diagonal matrix.
4. Identity matrix: The \(m \times m\) diagonal matrix with all diagonal entries equal to 1 is called the identity matrix and is denoted by \(I\) (or, \(I_m\)). It has the property that for any \(m \times n\) matrix \(A\) and any \(p x m\) matrix \(B\), \(IA = A\) and \(BI = B\).
5. One vector: The \(m \times 1\) vector with all entries equal to 1 is usually called the one vector (non-standard term) and is denoted by 1(or, \(1_m\)).
6. Ones matrix: The \(m \times m\) matrix with all entries equal to 1 is denoted by \(J\) (or, \(J_m\)). Note that \(J_m\) = \(1_m1_{m}^{T}\).
7. Zero vector: The \(m \times 1\) vector with all entries zero is called the zero vector and is denoted by \(0\) (or, \(0_m\)).

• Multiplication is not commutative: If \(A\) and \(B\) are both \(m \times m\) matrices then both \(AB\) and \(BA\) are defined and are \(m \times m\) matrices. However, in general \(AB \neq BA\). Notice that \(I_mB = BI_m = B\), where \(I_m\) is the identity matrix.
• Linear independence: The \(m \times 1\) vectors \(x_1,...,x_k\), (\(k\) arbitrary) are said to be linearly dependent, if there exist constants \(c_1,..., c_m\), not all zero, such that $$c_1x_1+c_2x_2 + ... + c_mx_m = 0$$ If no such sequence of numbers \(c1, ... , c_m\) exists then the vectors \(x_1, ..., x_m\) are said to be linearly independent.
  

1. Relationship with dimension: If \(k > m\) then \(m \times 1\) vectors \(x_1, ..., x_k\) are always linearly dependent.
2. Rank of a matrix: For an \(m \times n\) matrix \(A\), the rank of \(A\), written rank(\(A\)) is the maximal number of linearly independent columns of \(A\) (treating each column as an \(m \times 1\) vector). Also, rank(\(A\))\(\leq\)min\(\{m,n\}\)
3. Nonsingular matrix: If an \(m \times m\) matrix \(A\) has full rank, that is, rank(\(A\)) = \(m\), (which is equivalent to saying that all the columns of \(A\) are linearly independent), then the matrix \(A\) is called nonsingular

• Inverse of a matrix: If an \(m \times m\) matrix \(A\) nonsingular, then it has an inverse,
  that is a unique \(m \times m\) matrix denoted by \(A^{-1}\) that satisfies the relationship : \(A^{-1}A = I_m = AA^{-1}\)

Inverse of a \(2 \times 2\) matrix: Let a \(2 \times 2\) matrix \(A\) be expressed as \(A = \begin{bmatrix}
a &b \\
c &d
\end{bmatrix}\).

Then \(A\) is nonsingular (and hence has an inverse) if and only if \(ad-bc \neq 0\). If this is satisfied then the inverse is $$A^{-1} = \frac{1}{ad-bc}\begin{bmatrix}
d & -b\\
-c & a
\end{bmatrix}$$

2. Solution of a system of linear equations : A system of \(m\) linear equations in \(m\) variables \(b_1, ...,, b_m\) can be expressed as $$a_{1_1}b_1 + a_{1_2}b_2 + ... + a_{1_m}b_m = c_1$$ $$a_{2_1}b_1 + a_{2_2}b_2 + ... + a_{2_m}b_m = c_2$$ $$ ... ... ... ... ... = .$$ $$a_{m_1}b_1 + a_{m_2}b_2 + ... + a_{m_m}b_m = c_m $$

Here the coefficients \(a_{i_j}\) and the constants \(c_i\) are considered unknown. This system can be expressed in matrix form as \(Ab = c\), where \(A\) is the \(m \times m\) matrix with \((i, j)\)-th entry \(a_{i_j}\), and b and c are \(m \times 1\) vectors with \(i\)-th entries \(b_i\) and \(c_i\), respectively, for \(i = 1, ... , m; j = 1, ..., m\).

If the matrix \(A\) is nonsingular, then a unique solution exists for this system of equations and is given by b = \(A^{-1}c\). To see this, note that since \(A(A^{-1}) = (AA^{-1})c= I c= c\), it shows that \(A^{-1}c\) is a solution. On the other hand, if \(b = b^*\) is a solution, then it satisfies \(Ab^* = c\). Hence \(b^* = I b^* = (A^{-1}A)b^* = A^{-1}(Ab^*) = A^{-1}c\), which proves uniqueness.

Contributors

• Debashis Paul
• Cathy Wang