How to Perform Basic Matrix Operations

We introduce and define matrices. Furthermore, we learn how to perform matrix addition and matrix subtraction.

A matrix is simply a 2-dimensional vector. While vectors only have one dimension m, matrices have 2 dimensions m and n

A =
   \begin{bmatrix}
    a_{11} & ... & a_{1n}  \\
    ... & ... & ... \\
    a_{m1} & ... & a_{mn}  \\
    
  \end{bmatrix}

We say A is an m \times n matrix. As you can see, we usually denote matrices with capital letters.

Matrix Addition

Adding matrices A and B is achieved by obtaining the element-wise sum C of A and B. 

For matrix addition to work, both matrices need two have the same number of dimensions.

In math notation we would say:

A \in \mathbb{R}^{m \times n} + 
B \in \mathbb{R}^{m \times n} =
C \in \mathbb{R}^{m \times n}

For example:

 A + B =
 \begin{bmatrix}
    4 & 4 \\
    5 & 5 \\

  \end{bmatrix}

+

  \begin{bmatrix}
    1 & 2 \\
    1 & 2 \\

  \end{bmatrix}
=

  \begin{bmatrix}
    5 & 6 \\
    6 & 7 \\

  \end{bmatrix}

Note: In strict mathematical terms, you can only add two matrices if they have the same dimensions. If you do matrix math in a programming language like python, you’ll encounter a concept called broadcasting. This allows you to perform operations like matrix-vector addition. For example, if you add a   matrix A and a   dimensional vector b, python will replicate b 4 times, essentially creating a matrix B of the same dimensions of A for the purpose of addition.

Matrix Subtraction

According to the same rules, subtracting matrices A and B is possible by taking:

A \in \mathbb{R}^{m \times n} - 
B \in \mathbb{R}^{m \times n} =
C \in \mathbb{R}^{m \times n}

In our example this gives us the following result:

  
A - B =
\begin{bmatrix}
    4 & 4 \\
    5 & 5 \\

  \end{bmatrix}

-

  \begin{bmatrix}
    1 & 2 \\
    1 & 2 \\

  \end{bmatrix}
=
  \begin{bmatrix}
    3 & 2 \\
    4 & 3 \\

  \end{bmatrix}

Matrix Division?

I’ve included division only for completeness’s sake. Mathematically, matrix division is not defined. However, in cases where division would be required, we usually invert a matrix.

This post is part of a series on linear algebra for machine learning. To read other posts in this series, go to the index.