Log In to continue learning
Linear Algebra for Machine Learning and Data Science
34h5m
In this video, I will show you a way to tell if a matrix is singular or non-singular directly, without having to solve the system of linear equations. But first, let us look back at systems of sentences. Recall that a system of sentences is singular if the second sentence carries the same information as the first one. Similarly, a system of equations is singular if the second equation carries the same information as the first one. This is the concept of linear dependence. Let's look back at the two systems of linear equations that you've seen so far with their corresponding matrices. And let's focus on the singular system on the right. The reason this system is singular is because the second equation is a multiple of the first one. In particular, it's two times the first one. That means if we take the left hand and the right hand and multiply everything by two, in the first equation, you get the second equation. Now, if you look at the corresponding matrix, then the second row is a multiple of the first one. And by that, we mean that if you take every element in the first row and multiply them all by two, you get the second row. That means that the second row can be obtained from the first one. So the second row is dependent of the first one. Notice that you can also say that the first row is dependent of the second one by taking the second row and multiplying everything by a half. So in any way, either one depends on the other one or the other one depends on the one. They're both dependent, so they're linearly dependent. In contrast, in the non-singular system on the left, the second equation is not a multiple of the first one or vice versa. There's no constant that I can multiply the first equation to get the second equation or vice versa. This is why the system is non-singular. Each equation tells you something completely different. So in the corresponding matrix, the same thing happens. No row is a multiple of the other one. I cannot take a number and multiply one row entirely by the number to get the other row. That means the rows are linearly independent. As you may imagine, the same thing happens with columns and rows. One can define these concepts of linear dependency between rows and between columns. And they determine the singularity and non-singularity of the matrix. For a larger matrix, the concept of linear independence and dependence is a bit more complex. But still very intuitive. Follow along for the details. In order to understand linear dependence, consider the following example. Take a look at the system with three equations and three unknowns. The equations are A equals 1, B equals 2, and A plus B equals 3. Now, notice that C doesn't appear. But that doesn't matter. Now, this system is singular for the simple reason that the third equation is the sum of the first two. And to see this more in detail, the first equation can be written as A plus 0, B plus 0, C equals 1. The second one is 0, A plus B plus 0, C equals 2. And if we add them, we get A plus B plus 0, C equals 3, which is precisely the third equation. Now, consider the matrix of the coefficients of this system. That's the matrix obtained by forgetting the constants. The matrix has entries 1, 0, 0, 0, 1, 0, and 1, 1, 0. And in the same way that the sum of the first two equations is the third one, the sum of the first two rows in the matrix is equal to the third one. Because 1 plus 0 is 1, 0 plus 1 is 1, and 0 plus 0 is 0. Therefore, row 3 depends on rows 1 and 2. Row 3, in the same way as equation 3, doesn't bring anything new to the table. It's the sum of the first two. So we say that the rows are linearly dependent. Because you can get from the first and the second, you can form the third one. Therefore, this matrix and this system are singular. Now let's look at another system which you've already seen that it's singular. The corresponding matrix is this one over here, 1, 1, 1, 2, 2, 2, 3, 3, 3. Notice that there are many relations between the equations of this system. Here's one. The first equation plus the second equation equals the third equation. And in the matrix, row 1 plus row 2 is equal to row 3. So row 3 depends on rows 1 and 2, meaning that the rows are linearly dependent. Now if you look closely, this system and this matrix have a lot of other dependencies. For example, the second row is twice the first row, and the third row is three times the first row. This is a highly singular system with a lot of dependencies between the rows. Now in order to really understand the concept of linear dependence and independence, let's take a look at this other system which you've also seen is singular. Now this one is a little more subtle. What are the relations between the equations? Well, here's one. Let's take the first equation and let's add it to the third equation. What do you get? You get 2a plus 2b plus 4c is equal to 0. Now that doesn't look like one of the equations perhaps, or perhaps yes. If you take this equation and divide it by 2, you get a plus b plus 2c equals 0, which is precisely the second equation. So in other words, the second equation is the average of the first and third equation. And in the same way, the average between row 1 and row 3 of the matrix is row 2. So row 2 depends on rows 1 and 3. So therefore the rows of this matrix, as well as the equations in the system, are linearly dependent. Now in contrast, take a look at this system of equations. You've seen before that this system has a unique solution, so it's non-singular. Now no matter how much you try, it is impossible to find any linear relations between these equations. There's no way you can obtain the third one out of the other two, or the second one out of the first and the third. There's nothing you can do here. And the same thing happens in the matrix. No row can be obtained as a linear combination of the other two rows. Thus the rows are linearly independent. And this implies that the matrix is non-singular. Now as you can see, it's not easy to see and to verify that there's no relation. So don't worry, soon I'm going to show you some methods to be able to verify this. But for now, you're ready for a quiz. Determine if the following matrices have linearly dependent or independent rows. And the answers are in! For the first matrix, notice that the first row times 3 plus the second row times 2 is equal to the third row. And so the rows are linearly dependent. This implies that the matrix is singular. For the second matrix, notice that if you subtract the second row from the first one, you get the third row. Thus the rows are again linearly dependent, showing that the matrix is singular. The third matrix has no relations. Notice that no matter how much you try, there's no way to get one row out of the other two. For example, the first row has that leftmost 1. There's no way you can obtain that from linear combinations of the second and third row, and so on. So this matrix is non-singular because the rows are linearly independent. And finally, for the fourth matrix, notice that the first row times 2 is equal to the third row. So these rows are dependent, and the matrix is singular.