Matrices

For matrices, we just go over a few points.

A bit more intuition about parts of matrices will be added later, but for our purposes, it is enough to consider them just a tool to write down equations succinctly.

A matrix is a table of values. Matrices will be written capitalized in bold-face, for example $\mathbf{A}$.

A matrix has two dimensions, the number of rows and the number of cols. The number of rows is specified first.

The following is an example of a $3\times 2$ matrix:

$$\mathbf{A} = \begin{pmatrix}a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32}\end{pmatrix}$$

You can also usually see the following way of writing: $\mathbf{A}\in \mathbb{R}^{m\times n}$. This just means: "The matrix $\mathbf{A}$ is an element of the space of matrices of size $m\times n$ with real numbers as entries.

And a vector is just a matrix with one column, so a $n\times 1$ matrix!

You can transpose a matrix. This just means, you mirror the matrix at its diagonal. You can also just remember, that a matrix element $a_{ij}$ gets put at the position $a_{ji}$ (so the diagonal stays the same). The number of rows and columns is also swapped out by this operations.

The transpose of a matrix $\mathbf{A}$ is written as $\mathbf{A}^T$.

There are some additional ways to write a matrix, which sometimes make the intent clearer. Instead of specifying every single element, we could make a matrix out of a number of columns (each is a vector!):

$$\mathbf{A} = \begin{pmatrix} \mathbf{a}_1 & \dots & \mathbf{a}_n\end{pmatrix}$$

Similarly, we could write down the rows. To still use vectors, we need to convert a column to a row, which is what the transpose operation does.

$$\mathbf{A} = \begin{pmatrix} \mathbf{a}_1^T \\ \vdots \\ \mathbf{a}_m^T\end{pmatrix}$$

Addition and subtraction are performed element-wise. It makes sense then, that both matrices have to be the same size for that.

Multiplication is slightly more involved, but for now we will just write down one easy to remember version to write it.

First, a multiplication from the left side of a matrix $\mathbf{A}$ with a vector $\mathbf{b}$:

$$\begin{align*} \mathbf{A}\mathbf{b} &= \begin{pmatrix}\mathbf{a}_1^T \\ \vdots \\ \mathbf{a}_m^T\end{pmatrix}\mathbf{b} \\ &= \begin{pmatrix}\mathbf{a}_1 \cdot \mathbf{b} \\ \vdots \\ \mathbf{a}_m \cdot \mathbf{b}\end{pmatrix} \end{align*}$$

The result will be a vector with $m$ rows. For the dot products to make sense, $\mathbf{A}$ needs to have $n$ columns, where $n$ is the size of the vector.

With that, you can also calculate the product of two matrices, by just thinking about the right matrix as a number of columns.

$$\begin{align*} \mathbf{A}\mathbf{B} &= \begin{pmatrix}\mathbf{a}_1^T \\ \vdots \\ \mathbf{a}_m^T\end{pmatrix} \begin{pmatrix}\mathbf{b}_1 & \dots &\mathbf{b}_n\end{pmatrix} \\ &= \begin{pmatrix}\mathbf{A}\mathbf{b}_1 & \dots & \mathbf{A} \mathbf{b}_n\end{pmatrix}\\ &= \begin{pmatrix}\mathbf{a}_1 \cdot \mathbf{b}_1 & \dots & \mathbf{a}_1 \cdot \mathbf{b}_n \\ \vdots& & \vdots \\ \mathbf{a}_m \cdot \mathbf{b}_1 &\dots & \mathbf{a}_m \cdot \mathbf{b}_n \end{pmatrix} \end{align*}$$

For this to work, the columns of the first matrix need to match the rows of the second. A $m\times p$ matrix times a $p \times n$ matrix results in a $m\times n$ matrix.

This means, that not every matrix can be multiplied with any other one. This also means, we can't in general switch the order of multiplications. Even if the dimensions match, in general the following holds:

$$\mathbf{A}\mathbf{B} \neq \mathbf{B}\mathbf{A}$$

This will be important when transforming objects in space.

We can also see from this notation, that the dot product seems to have some kind of connection to this. Sometimes, it is useful to "convert" a dot product of two vectors to matrix notation.

$$\begin{aligned} \mathbf{a} \cdot \mathbf{b} &= \mathbf{a}^T \mathbf{b} \end{aligned}$$

This multiplies a $1\times n$ matrix with a $n \times 1$ matrix, resulting in a $1 \times 1$ matrix, so just a number!

One special matrix is the so called identity matrix. We will write it as $\mathbf{I}_n$. This matrix is square and has dimension $n$, so it is a $n \times n$ matrix. If the dimension doesn't matter, we just write $\mathbf{I}$.

It has 1s on the diagonal and 0s everywhere else.

If the dimensions match, the identity matrix leaves any matrix unmodified (like multiplying by $1$):

$$\mathbf{I}\mathbf{A} = \mathbf{A}\mathbf{I} = \mathbf{A}$$

With numbers, we can write $a \frac{1}{a} = 1$. We say that $\frac{1}{a}$ is the inverse of $a$ which when multiplied makes it $1$.

For matrices we have something similar and it is also called inverse. When multiplied with a matrix, it results in the identity matrix. We write the inverse of $\mathbf{A}$ as $\mathbf{A}^{-1}$.

$$\mathbf{A}\mathbf{A}^{-1} = \mathbf{A}^{-1}\mathbf{A} = \mathbf{I}$$

For this to be defined properly, it only exists for square matrices. And similar to numbers, where $0$ does not have an inverse, there are matrices that do not have an inverse.

Sometimes, you might also see the notation $\mathbf{A}^{-T}$, which is just a shorthand for $(\mathbf{A}^{-1})^T = (\mathbf{A}^{T})^{-1}$. Both sides of the equation are equivalent.

We don't need to consider the specifics here.

This concludes our very brief overview of the necessary information about matrices.