How to Perform Basic Matrix Operations

Posted by Seb On November 22, 2020 In Mathematics for Machine Learning

Sharing is caring

We introduce and define matrices. Furthermore, we learn how to perform matrix addition and matrix subtraction.

A matrix is simply a 2-dimensional vector. While vectors only have one dimension m, matrices have 2 dimensions m and n

A = \begin{bmatrix} a_{11} & ... & a_{1n} \\ ... & ... & ... \\ a_{m1} & ... & a_{mn} \\ \end{bmatrix}

We say A is an $m \times n$ matrix. As you can see, we usually denote matrices with capital letters.

Matrix Addition

Adding matrices A and B is achieved by obtaining the element-wise sum C of A and B.

For matrix addition to work, both matrices need two have the same number of dimensions.

In math notation we would say:

A \in \mathbb{R}^{m \times n} + B \in \mathbb{R}^{m \times n} = C \in \mathbb{R}^{m \times n}

For example:

A + B = \begin{bmatrix} 4 & 4 \\ 5 & 5 \\ \end{bmatrix} + \begin{bmatrix} 1 & 2 \\ 1 & 2 \\ \end{bmatrix} = \begin{bmatrix} 5 & 6 \\ 6 & 7 \\ \end{bmatrix}

Note: In strict mathematical terms, you can only add two matrices if they have the same dimensions. If you do matrix math in a programming language like python, you’ll encounter a concept called broadcasting. This allows you to perform operations like matrix-vector addition. For example, if you add a matrix A and a dimensional vector b, python will replicate b 4 times, essentially creating a matrix B of the same dimensions of A for the purpose of addition.

Matrix Subtraction

According to the same rules, subtracting matrices A and B is possible by taking:

A \in \mathbb{R}^{m \times n} - B \in \mathbb{R}^{m \times n} = C \in \mathbb{R}^{m \times n}

In our example this gives us the following result:

A - B = \begin{bmatrix} 4 & 4 \\ 5 & 5 \\ \end{bmatrix} - \begin{bmatrix} 1 & 2 \\ 1 & 2 \\ \end{bmatrix} = \begin{bmatrix} 3 & 2 \\ 4 & 3 \\ \end{bmatrix}

Matrix Division?

I’ve included division only for completeness’s sake. Mathematically, matrix division is not defined. However, in cases where division would be required, we usually invert a matrix.

This post is part of a series on linear algebra for machine learning. To read other posts in this series, go to the index.