Understanding the Determinant of a Matrix

We introduce the matrix determinant and its uses. Furthermore, we learn how the determinant provides us with a shortcut to finding the inverse matrix.

What is the matrix determinant?

The determinant is a scalar value that can be obtained from a square matrix and which can be used to find the inverse of a matrix.

For some matrices, the process of finding the inverse can be fairly tricky and we usually rely on computers to do it for us.
It turns out that for some square matrices, there is a faster way, using the matrix determinant, to find the inverse and to determine whether we are dealing with an invertible matrix at all.

Finding the Inverse Matrix

If you have a square matrix A the inverse can be obtained through simple algebraic manipulation. We start by first inverting the terms on the leading diagonal and putting a minus in front of the terms on the off-diagonal which results in a matrix we call A*.

A = 
  \begin{bmatrix}
    a & b \\
    c & d \\
  \end{bmatrix}

A^{*} = 
  \begin{bmatrix}
    d & -b \\
    -c & a \\
  \end{bmatrix}

When you multiply A with its A*, you arrive at the following matrix.:

  \begin{bmatrix}
    a & b \\
    c & d \\
  \end{bmatrix}
  \begin{bmatrix}
    d & -b \\
    -c & a \\
  \end{bmatrix}
=
  \begin{bmatrix}
    ad - bc & 0 \\
    0 & ad - bc \\
  \end{bmatrix}

The term ad – bc is known as the determinant of a matrix. Using the determinant and A* we can find the inverse of A. We take 1 over the determinant and perform scalar multiplication, which results in the identity matrix.

  \frac{1}{ad - bc}
\begin{bmatrix}
    ad - bc & 0 \\
    0 & ad - bc \\
  \end{bmatrix}

= \begin{bmatrix}
    1 & 0 \\
    0 & 1 \\
  \end{bmatrix} =
I

Accordingly, the following expression must also result in the identity matrix.

  \frac{1}{ad - bc}
\begin{bmatrix}
    a & b \\
    c & d \\
  \end{bmatrix}
  \begin{bmatrix}
    d & -b \\
    -c & a \\
  \end{bmatrix}

= 
I

The expression whose multiplication with A results in the identity matrix is equivalent to the inverse of A.

  \frac{1}{ad - bc}  
\begin{bmatrix}
    d & -b \\
    -c & a \\
  \end{bmatrix}
= A^{-1}

Please remember, a determinant only exists if the matrix has the same number of rows as it has columns.

Is A an Invertible Matrix?

We can only obtain the inverse of matrix A if A is, in fact, invertible. Luckily, there is an easy way to figure this out. A is only invertible if the following condition holds.

ad - bc \neq 0

Why does this hold?

Imagine you had the following matrix.

A =
\begin{bmatrix}
    a & b \\
    c & d \\
  \end{bmatrix} 
=
 \begin{bmatrix}
    1 & 0 \\
    0 & 1 \\
  \end{bmatrix}
\, 
consisting \, of \,
a_1 = \begin{bmatrix}
    1 \\
    0 \\
  \end{bmatrix}
a_2 = \begin{bmatrix}
    0 \\
    1 \\
  \end{bmatrix}

The columns of the matrix are vectors that span a plane. The determinant of a matrix is a measure of the area of that plane.

ad - bc = 1\times1 - 0\times 0 = 1

If b and c were non-zero numbers, then the plane would be stretched and the area would change. I won’t go into the geometry, but you can check this for yourself.

Imagine you have a matrix consisting of two vectors.

A =
\begin{bmatrix}
    a & b \\
    c & d \\
  \end{bmatrix} 
=
 \begin{bmatrix}
    2 & 4 \\
    2 & 4 \\
  \end{bmatrix}
\, 
consisting \, of \,
a_1 = \begin{bmatrix}
    2 \\
    2 \\
  \end{bmatrix}
a_2 = \begin{bmatrix}
    4 \\
    4 \\
  \end{bmatrix}

If you calculate the determinant, it is zero

ad - bc = 2\times4 - 4\times 2 = 0

As you can see in the plot, a2 is just an extension of a1.

So a1 and a2 do not span a plane or a volume. There is no discernible area and therefore the determinant is zero. You also won’t be able to determine the inverse of that matrix, because you cannot divide by zero (remember, that earlier we divided 1 by the determinant of A to determine the inverse of the matrix A).

Determinant of a 3×3 Matrix

So far we’ve looked at the determinant of a 2×2 matrix. When we move into 3 dimensional space, the determinant helps us determine the volume of the object that the columns span.

For a 3D matrix A, the determinant is defined as follows.

A = \begin{bmatrix}
    a & b & c \\
    d & e & f \\
    g & h & i \\
  \end{bmatrix} 
\, then

|A| = a(ei - fh) - b(di - fg) + c(dh - fg)

At first sight this look very confusing. The rule is to multiply each element in the first row, with the determinant of the 2×2 matrix of the elements that are not in the same row or column as the first element.

I am not going to go through an example. because frankly, it isn’t necessary to do this manually anymore. Any computer programming language used to manipulate matrices, has a function to calculate the determinant. Furthermore, it does not affect the conceptual understanding of the other ideas.

This post is part of a series on linear algebra for machine learning. To read other posts in this series, go to the index.