Materials covered in Part 2:

In this article, I’m going to cover important fundamental concepts in Linear Algebra such as:

• Linear Combinations
• Column and Row space
• Nullspace (or sometimes referred to as Kernel)
• Rank
• Pivot variables and Free variables

If you know all the above contents, you are good to go. I have a summary of these at the bottom for you to recap these key concepts quickly.

Recapping linear combinations

I talked a little about linear combinations in Part 1, but let’s make this concept stick.

Linear combination is any combination that can be expressed with

1) multiplying with constants

2) adding together

3) the combinations of 1 and 2

For example, let’s say you have 2 column vectors v

It could be any combinations as long as they are linearly combined together. In other words, you are not transforming the vectors such as multiplying the vectors together. If you want to know a little bit more, check the link below.

Introduction to row and column space

So far, we studied vector space and subspace. Let’s talk about column and row space.

By now, you should know what’s rows and columns.

Long story short,

• row space is a subspace that is represented by a linear combinations of row vectors
• column space is a subspace that is represented by a linear combinations of column vectors

Let’s take a look at an easy example. Suppose you had a matrix like the one below. If that’s the case, we have 2 column vectors. The column space represented by such vectors are shown as below. It’s like a 45 degrees tilted plane on x-axis.

Same goes for the row space. I embedded a link to the wiki regarding the topic, so feel free to jump off and take a look below.

Introduction to nullspace

Now comes a very important concept in Linear Algebra called nullspace! Nullspace is a subspace represented by the following equation:

Notice that we are not solving the quadratic equations in the end? Let’s try that and see what happens.

This leads us nowhere. We want answers with non-zeros to understand what kind of x we are looking here.

So by choosing the arbitrary numbers, we have 2 resulting vectors. This will create a nullspace like the one below. Since we are looking at a 2D space, and there’s two vectors, the nullspace is basically the whole plane covering the 2D plane.

To summarize, nullspace is a subspace represented by the set of all vectors x that satisfies Ax = 0 (Given that A is a matrix, 0 is a zero vector).

Introduction to rank

Suppose you have a matrix like the one below.

Rank of a matrix is equivalent to the number of pivots. So let’s perform row elimination to find out pivots.

In this matrix, we have 2 pivots. The rank is the same as the number of pivots. Therefore, we have rank(A) = 2.

However, did you notice that the matrix A has 3 columns? So we have the matrix rank = 2, and the matrix dimension = 3. Let’s see where this leads us. First, we convert the row echelon form to reduced row echelon form like what we did in Part 1.

And let’s try to solve this based on the nullspace equation: Ax = 0

Now you can see there’s something odd here. Dimensionality of the matrix tells us how many variables x we need to identify, but since the rank of the matrix is 2, which is 1 less than the dimension, we only have 2 equations for 3 variables. If you remember back in the old days when you were solving quadratic equations, you know that you can’t solve the equations because we just don’t have enough information.

Let me give you another example to prove this point. Let’s think of another matrix that has the dimension of 3 and the rank of 3.

So now by looking at this comparison, I hope you understand that there’s no way to figure out the exact x_2 on the left example where the rank is 2 and 1 less than the dimension.

One last concept today is about free variables. In the left example above, we have no way to figure out the exact x_2 and if that’s the case, we can literally assign any values. It could be 1, 0.5, or 92842, etc. Usually we assign the most basic ones like 1. We call these variables, free variables. Important thing to remember is that the number of free variables corresponds to the dimension minus the rank. In the example above, we had dimension = 3 and rank = 2 which gives us 3–2= 1. That’s why we only had 1 free variable in the example.

Like I said in the Part 1, feel free to google and search for other explanations and materials if you still feel like you are missing something or the concepts are still hanging over your head. I’m trying to give you an initial trigger to know the material. To understand the concept, you need to study, process the information, until you get the concept right.

Summary

Let’s summarize what we have learned in this article:

• Linear Combinations

Any combinations of 1) multiplying with constants and 2) summing the vectors are called linear combinations.

• Column and Row space

A subspace represented by a linear combinations of column vectors or row vectors.

• Nullspace (or sometimes referred to as Kernel)

A subspace represented by all the linear combinations of x, where x satisfies: Ax = 0.

• Rank

The rank of a matrix is equivalent to the number of pivots of the matrix.

• Pivot variables and Free variables

The sum of the number of pivots and free variables equals to the dimension of the matrix. In other words, the number of free variables = dimension of a matrix minus the rank.