Multiplication of Matrices

2.4. Multiplication of Matrices#

Row and Column Vectors

Column Matrices (Vectors): A column matrix, also known as a column vector, is a matrix that has only one column and any number of rows. A column matrix can be written as

\[\begin{align*} A = \begin{bmatrix} a_{1} \\ a_{2} \\ \vdots \\ a_{n} \end{bmatrix} \end{align*}\]

where $n$ is the number of rows and $a_i$ is the entry in the $i^{th}$ row. A column matrix has size $n \times 1$, which means it has $n$ rows and one column. Column matrices are important in linear algebra because they can represent geometric vectors, which are quantities that have both magnitude and direction. Geometric vectors can be used to model physical phenomena such as forces, velocities, displacements, and more [Kuttler and Farah, 2020, Nicholson, 2018].

Row Matrices (Row Vectors): A row matrix, also known as a row vector, is a matrix that has only one row and any number of columns. A row matrix can be written as

\[\begin{align*} B = \begin{bmatrix} b_{1} & b_{2} & \dots & b_{n} \end{bmatrix} \end{align*}\]

where $n$ is the number of columns and $b_i$ is the entry in the $i^{th}$ column. A row matrix has size $1 \times n$, which means it has one row and $n$ columns. Row matrices are also related to geometric vectors, but they are usually written as transposes of column matrices. That is, if $A$ is a column matrix, then $B=A^T$ is a row matrix, where $T$ denotes the transpose operation. The transpose of a matrix is obtained by flipping the rows and columns of the matrix [Kuttler and Farah, 2020, Nicholson, 2018].

Example: For example, if $A$ is a column matrix of size $3 \times 1$, it would look like this:

\[\begin{align*} A = \begin{bmatrix} a_{1} \\ a_{2} \\ a_{3} \end{bmatrix} \end{align*}\]

Likewise, if $B$ is a row matrix of size $1 \times 4$, it would look like this:

\[\begin{align*} B = \begin{bmatrix} b_{1} & b_{2} & b_{3} & b_{4} \end{bmatrix} \end{align*}\]

Both column matrices (vectors) and row matrices (row vectors) play a crucial role in linear algebra, and they are used to represent quantities such as position, velocity, and forces in physics, as well as data in various applications like machine learning and statistics.

Before we explain multiplying matrices. Let’s explain the multiplication of a row vector by a column vector. For row vector $X$ and a column vector $Y$. we have,

\[\begin{align*} \begin{bmatrix} x_1 & x_2 & \dots & x_n \end{bmatrix} \begin{bmatrix} y_1\\ y_2 \\ \vdots \\ y_n \\ \end{bmatrix}= x_1y_1+x_2y_2+\ldots+x_ny_n=\sum_{j=1}^{n}x_jy_j.\end{align*}\]

For a row vector $X$ and a column vector $Y$, where both have the same size $n \times 1$, we perform the multiplication as follows:

\[\begin{align*} X = \begin{bmatrix} x_1 & x_2 & \dots & x_n \end{bmatrix} \end{align*}\]

\[\begin{align*} Y = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} \end{align*}\]

The multiplication of the row vector $X$ and column vector $Y$ is given by the sum of the products of their corresponding entries:

\[\begin{align*} \begin{bmatrix} x_1 & x_2 & \dots & x_n \end{bmatrix} \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} = x_1y_1 + x_2y_2 + \ldots + x_ny_n \end{align*}\]

This can be represented using summation notation as:

\[\begin{align*} \begin{bmatrix} x_1 & x_2 & \dots & x_n \end{bmatrix} \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_n \end{bmatrix} = \sum_{j=1}^{n} x_j y_j \end{align*}\]

Where:

$x_j$ represents the $j$-th entry in the row vector $X$.
$y_j$ represents the $j$-th entry in the column vector $Y$.
The summation symbol $\sum$ denotes the sum of the products of the corresponding entries, ranging from $j=1$ to $j=n$.

In other words, we multiply each entry in the row vector $X$ by the corresponding entry in the column vector $Y$, and then sum up these products to get the final result.

Example: For example, if we have:

\[\begin{align*} X = \begin{bmatrix} 2 & 3 & 4 \end{bmatrix} \end{align*}\]

\[\begin{align*} Y = \begin{bmatrix} 1 \\ -1 \\ 2 \end{bmatrix} \end{align*}\]

The multiplication of the row vector $X$ and column vector $Y$ is:

\[\begin{align*} \begin{bmatrix} 2 & 3 & 4 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \\ 2 \end{bmatrix} = (2 \cdot 1) + (3 \cdot (-1)) + (4 \cdot 2) = 2 - 3 + 8 = 7 \end{align*}\]

Example: Evaluate $\begin{bmatrix} 1 & 2 & 3 \end{bmatrix}\begin{bmatrix} -2 \\ 1 \\ 4 \end{bmatrix}$.

Solution:

We have the row vector:

\[\begin{align*} X = \begin{bmatrix} 1 & 2 & 3 \end{bmatrix} \end{align*}\]

And the column vector:

\[\begin{align*} Y = \begin{bmatrix} -2 \\ 1 \\ 4 \end{bmatrix} \end{align*}\]

To multiply the row vector $X$ by the column vector $Y$, we follow these steps:

Take the first entry of the row vector ($1$) and the first entry of the column vector ($-2$), then multiply them together: $(1) \times (-2) = -2$.
Take the second entry of the row vector ($2$) and the second entry of the column vector ($1$), then multiply them together: $(2) \times (1) = 2$.
Take the third entry of the row vector ($3$) and the third entry of the column vector ($4$), then multiply them together: $(3) \times (4) = 12$.

Finally, add up the results of these individual multiplications:

\[\begin{align*} (-2) + 2 + 12 = 12 \end{align*}\]

So, the result of multiplying the row vector $X$ and the column vector $Y$ is $12$.

In summary, the multiplication of a row vector by a column vector is obtained by taking the dot product of the vectors. It involves multiplying each corresponding pair of entries and then summing up these products. The dot product is a common operation in linear algebra and has various applications in vector calculations and geometry.

Remark

To multiply two matrices $A$ and $B$, we need to make sure that the number of columns of $A$ matches the number of rows of $B$. This is because we multiply each element in a row of $A$ with the corresponding element in a column of $B$, and then add them up to get an entry in the product matrix $AB$. For example, if $A$ has size $m\times n$ and $B$ has size $n\times p$, then the product $AB$ will have size $m\times p$.

Notice that the inner dimensions of $A$ and $B$ are both $n$, which means they are compatible for multiplication. The outer dimensions of $A$ and $B$ are $m$ and $p$, which determine the size of the product $AB$. We can write this as

\[\begin{align*} (m\times \overbrace{n) (n}^{\text{these must match!}} \times p ) = m\times p \end{align*}\]

If the number of columns of $A$ is not equal to the number of rows of $B$, then the product $AB$ is undefined, and we cannot perform the multiplication.

We explain this using an example:

Example: Let $A=\begin{bmatrix}3 & -1 \\ 0 & 2\end{bmatrix}$ and $B=\begin{bmatrix}2 & 4 \\ 1 & 7\end{bmatrix}$. Evaluate $AB$.

Solution: $A$ is a $2\times 2$ matrix, and $B$ is a $2\times 2$ matrix. Therefore, $AB$ is $2\times 2$ matrix as well. Evaluating each entry of $AB$:

$(1,~1)$-entry of a matrix $AB$: We multiply row $1$ of matrix $A$ and column $1$ of matrix $B$ (It is similar to multiplying a row vector by a column vector).

\[\begin{align*} \left[\begin{array}{cc}3 & -1\\ 0 & 2\end{array}\right] \left[\begin{array}{>{\columncolor{blue!10}}cc}2 & 4 \\ 1 & 7\end{array}\right]\end{align*}\]

\[\begin{align*} \text{$(1,~1)$-entry of a matrix $AB$}=\begin{bmatrix}3 & -1\end{bmatrix}\begin{bmatrix}2 & 1\end{bmatrix}=(3)(2)+(-1)(1)=3+2=5.\end{align*}\]
$(1,~2)$-entry of a matrix $AB$: We multiply row $1$ of matrix $A$ and column $2$ of matrix $B$.

\[\begin{align*} \left[\begin{array}{cc}3 & -1\\0 & 2\end{array}\right] \left[\begin{array}{c>{\columncolor{blue!10}}c}2 & 4 \\ 1 & 7\end{array}\right]\end{align*}\]

\[\begin{align*} \text{$(1,~2)$-entry of a matrix $AB$}=\begin{bmatrix}3 & -1\end{bmatrix}\begin{bmatrix}4 & 7\end{bmatrix}=(3)(4)+(-1)(7)=12-7=5.\end{align*}\]
$(2,~1)$-entry of a matrix $AB$: We multiply row $2$ of matrix $A$ and column $1$ of matrix $B$.

\[\begin{align*} \left[\begin{array}{cc}3 & -1\\ 0 & 2\end{array}\right] \left[\begin{array}{>{\columncolor{blue!10}}cc}2 & 4 \\ 1 & 7\end{array}\right]\end{align*}\]

\[\begin{align*} \text{$(2,~1)$-entry of a matrix $AB$}=\begin{bmatrix}0 & 2\end{bmatrix}\begin{bmatrix}2 & 1\end{bmatrix}=(0)(2)+(2)(1)=0+2=2.\end{align*}\]
$(2,~2)$-entry of a matrix $AB$: We multiply row $2$ of matrix $A$ and column $2$ of matrix $B$.

\[\begin{align*} \left[\begin{array}{cc}3 & -1\\ 0 & 2\end{array}\right] \left[\begin{array}{c>{\columncolor{blue!10}}c}2 & 4 \\ 1 & 7\end{array}\right]\end{align*}\]

\[\begin{align*} \text{$(2,~1)$-entry of a matrix $AB$}=\begin{bmatrix}0 & 2\end{bmatrix}\begin{bmatrix}1 & 7\end{bmatrix}=(0)(1)+(2)(7)=0+14=14.\end{align*}\]

Therefore, $AB=\begin{bmatrix}5 & 5\\ 2 & 14\end{bmatrix}$.

Example: Compute $AB$ where $A=\begin{bmatrix}1 & 2 \\ -3 & 0 \\ 1 & -4 \end{bmatrix}$ and $B=\begin{bmatrix} 1 & -1 & 2 \\ 3 & -2 & 1 \end{bmatrix}$.

Solution: $A$ is a $3\times 2$ matrix, and $B$ is a $2\times 3$ matrix. Therefore, $AB$ is $3\times 3$ matrix.
Evaluating the first row of $AB$:

\[\begin{align*} \left[\begin{array}{cc}1 & 2 \\ -3 & 0 \\ 1 & -4 \end{array}\right] \left[\begin{array}{>{\columncolor{blue!15}}c>{\columncolor{yellow!30}}c>{\columncolor{cyan!20}}c} 1 & -1 & 2 \\ 3 & -2 & 1 \end{array}\right] &= \begin{bmatrix} (1)(1)+(2)(3) & (1)(-1)+(2)(-2) & (1)(2)+(2)(1)\\ \centerdot & \centerdot & \centerdot \\ \centerdot & \centerdot & \centerdot\end{bmatrix} \\ &= \left[\begin{array}{ccc} 7 & -5 & 4\\ \centerdot & \centerdot & \centerdot \\ \centerdot & \centerdot & \centerdot \end{array}\right]\end{align*}\]

Evaluating the second row of $AB$:

\[\begin{align*} \left[\begin{array}{cc}1 & 2 \\ -3 & 0 \\ 1 & -4 \end{array}\right] \left[\begin{array}{>{\columncolor{blue!15}}c>{\columncolor{yellow!30}}c>{\columncolor{cyan!20}}c} 1 & -1 & 2 \\ 3 & -2 & 1 \end{array}\right] &= \begin{bmatrix} 7 & -5 & 4\\ (-3)(1)+(0)(3) & (-3)(-1)+(0)(-2) & (-3)(2)+(0)(1) \\ \centerdot & \centerdot & \centerdot\end{bmatrix} \\ &= \left[\begin{array}{ccc} 7 & -5 & 4\\ -3 & 3 & -6 \\ \centerdot & \centerdot & \centerdot \end{array}\right]\end{align*}\]

Evaluating the third row of $AB$:

\[\begin{align*} \left[\begin{array}{cc}1 & 2 \\ -3 & 0 \\ 1 & -4 \end{array}\right] \left[\begin{array}{>{\columncolor{blue!15}}c>{\columncolor{yellow!30}}c>{\columncolor{cyan!20}}c} 1 & -1 & 2 \\ 3 & -2 & 1 \end{array}\right] &= \begin{bmatrix} 7 & -5 & 4\\ -3 & 3 & -6 \\ (1)(1)+(-4)(3) & (1)(-1)+(-4)(-2) & (1)(2)+(-4)(1) \end{bmatrix} \\ &= \left[\begin{array}{ccc} 7 & -5 & 4\\ -3 & 3 & -6 \\ -11 & 7 & -2 \end{array}\right]\end{align*}\]

Therefore, $AB=\begin{bmatrix} 7 & -5 & 4\\ -3 & 3 & -6\\ -11 & 7 & -2 \end{bmatrix}$.

Remark

Matrix multiplication is a special operation that does not follow the usual rules of arithmetic. One of the most important differences is that matrix multiplication is not commutative, which means that changing the order of the factors can change the result. In other words, for two matrices $A$ and $B$, we cannot assume that $AB=BA$. In fact, most of the time, $AB$ and $BA$ are different matrices, or even undefined if the sizes do not match.

However, there are some special cases where matrix multiplication is commutative, and $AB=BA$ for any two matrices $A$ and $B$. These cases are rare and have certain properties that make them different from the general case. For example, if $A$ and $B$ are both scalar matrices, which are matrices that have the same number on the main diagonal and zero everywhere else, then $AB=BA$. This is because scalar matrices act like real numbers when multiplied by other matrices, and real numbers are commutative.

To illustrate this, let $A=\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}$ and $B=\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$. Then we can see that $A$ is a scalar matrix, and $B$ is any matrix of the same size. We can compute the products $AB$ and $BA$ as follows:

\[\begin{split} AB=\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} = \begin{bmatrix} 2\times 1 & 2\times 2 \\ 2\times 3& 2\times 4 \end{bmatrix} = \begin{bmatrix} 2 & 4 \\ 6& 8 \end{bmatrix} \end{split}\]

\[\begin{split} BA=\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix} = \begin{bmatrix} 1\times 2 & 2\times 2 \\ 3\times 2& 4\times 2 \end{bmatrix} = \begin{bmatrix} 2 & 4 \\ 6& 8 \end{bmatrix} \end{split}\]

We can see that $AB=BA$ in this case, because multiplying by $A$ is equivalent to multiplying by $2$, and multiplying by $2$ is commutative. However, this is not true for most matrices, and we should always be careful when multiplying matrices.