Skip to main content
\(\newcommand{\trace}{\operatorname{tr}} \newcommand{\real}{\operatorname{Re}} \newcommand{\imaginary}{\operatorname{Im}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

Section3.1Linear Algebra in a Nutshell

Linear algebra and matrices provide a convenient notation for representing the \(2 \times 2\) system \begin{align*} \frac{dx}{dt} & = a x + b y,\\ \frac{dy}{dt} & = c x + d y.\text{.} \end{align*} If we let \begin{equation*} A = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \quad\text{and}\quad {\mathbf x}(t) = \begin{pmatrix} x(t) \\ y(t) \end{pmatrix}, \end{equation*} then we can rewrite our system as \begin{equation*} \begin{pmatrix} x'(t) \\ y'(t) \end{pmatrix} = \begin{pmatrix} ax(t) + b y(t) \\ cx(t) + d y(t) \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} x(t) \\ y(t) \end{pmatrix}. \end{equation*} In other words, we can write our system as \begin{equation*} \frac{d \mathbf x}{dt} = A {\mathbf x}, \end{equation*} where \begin{equation*} \mathbf x' = \frac{d \mathbf x}{dt} = \begin{pmatrix} x'(t) \\ y'(t) \end{pmatrix}. \end{equation*}

Subsection3.1.1Matrices and Systems of Linear Equations

A short review of linear algebra and \(2 \times 2\) matrices is useful at this point. Recall that any system of two equations in two variables, \begin{align*} a x + by & = \alpha,\\ cx + dy & = \beta, \end{align*} can be written as a matrix equation \begin{equation} \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} ax + by \\ cx + dy \end{pmatrix} = \begin{pmatrix} \alpha \\ \beta \end{pmatrix}.\label{equation-linear01-2x2-3}\tag{3.1} \end{equation} We will denote the \(2 \times 2\) coefficient matrix by \(A\text{.}\) That is, \begin{equation*} A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}. \end{equation*}

If a solution for the system (3.1) exists, it is easy to find. A unique solution will occur exactly when the matrix \(A\) is invertible (or nonsingular). The unique solution is given by \begin{equation*} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} \begin{pmatrix} \alpha \\ \beta \end{pmatrix}, \end{equation*} where \begin{equation*} A^{-1} = \frac{1}{ad - bc} \begin{pmatrix} d & -b \\ -c & a \end{pmatrix}. \end{equation*} The matrix \(A\) is invertible if and only if its determinant is nonzero, \begin{equation*} \det(A) = ad - bc \neq 0. \end{equation*} If \(\det(A) = 0\text{,}\) then we either have no solution or infinitely many solutions.

Let us consider the special case \begin{equation*} A \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}. \end{equation*} If \(\det(A) \neq 0\text{,}\) we have exactly one solution, \(x = 0\) and \(y = 0\text{.}\) On the other hand, if \(\det(A) = 0\text{,}\) we have infinitely many solutions. Suppose that \(a \neq 0\text{.}\) Then \(x = - (b/a)y\text{,}\) and \begin{equation*} -c \left( \frac{b}{a}\right) y + dy = 0. \end{equation*} Therefore, \((ad - bc) y =0\text{.}\) Since \(\det(A) = ad - bc = 0\text{,}\) the variable \(y\) can assume any value and \(x = - (b/a)y\text{.}\) Thus, the solutions to our system lie along a line through the origin. In fact, we will always get a line of solutions through the origin as long as at least one entry in our matrix is nonzero. 14 

Subsection3.1.2Linear Independence

We say that two vectors \({\mathbf x}\) and \({\mathbf y}\) in \({\mathbb R}^2\) are linearly independent if they do not lie on the same line through the origin. If, on the other hand, they do lie on the same line, then the vectors are linearly dependent. Equivalently, two vectors are linearly dependent if one vector is a multiple of the other. We leave the proof of the following theorem as an exercise.

If we have a pair of linearly independent vectors in \({\mathbb R}^2\text{,}\) then we can write any vector in \({\mathbb R}^2\) as a unique linear combination of the two vectors. That is, given two linearly independent vectors \({\mathbf x} = (x_1, x_2)\) and \({\mathbf y} = (y_1, y_2)\text{,}\) we can write \({\mathbf z} = (z_1, z_2)\) as \begin{equation*} \begin{pmatrix} z_1 \\ z_2 \end{pmatrix} = \alpha \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} + \beta \begin{pmatrix} y_1 \\ y_2 \end{pmatrix}, \end{equation*} where \(\alpha\) and \(\beta\) are unique. To see why this is true, we must solve the equations \begin{align*} z_1 & = \alpha x_1 + \beta y_1\\ z_2 & = \alpha x_2 + \beta y_2 \end{align*} for \(\alpha\) and \(\beta\text{.}\) However, this system has a unique solution since \begin{equation*} \det \begin{pmatrix} x_1 & y_1 \\ x_2 & y_2 \end{pmatrix} \neq 0. \end{equation*} Two vectors are said to be a basis for \({\mathbb R}^2\) if we can write any vector in \({\mathbb R}^2\) as a linear combination of these two vectors. By our arguments above, any two linearly independent vectors will form a basis for \({\mathbb R}^2\text{.}\)

Example3.2

The vectors \({\mathbf e}_1 = (1, 0)\) and \({\mathbf e}_2 = (0, 1)\) form a basis for \({\mathbb R}^2\text{.}\) Indeed, if \({\mathbf z} = (z_1, z_2)\text{,}\) then we can write \begin{equation*} {\mathbf z} = z_1 {\mathbf e}_1 + z_2 {\mathbf e}_2. \end{equation*} The vectors \({\mathbf e}_1\) and \({\mathbf e}_2\) are called the standard basis for \({\mathbb R}^2\text{.}\)

Example3.3

Let \({\mathbf v}_1 = (2,1)\) and \({\mathbf v}_2 = (3, 2)\text{.}\) Since \begin{equation*} \det \begin{pmatrix} 2 & 3 \\ 1 & 2 \end{pmatrix} \neq 0, \end{equation*} these vectors form a basis for \({\mathbb R}^2\text{.}\) If \({\mathbf z} = (-5, -4)\text{,}\) then we can write \begin{equation*} {\mathbf z} = 2 {\mathbf v}_1 - 3 {\mathbf v}_2. \end{equation*} We say that the coordinates of \({\mathbf z}\) are \((2, -3)\) with respect to the basis \(\{ {\mathbf v}_1, {\mathbf v}_2 \}\text{.}\)

Example3.4

The vectors \((1, 1)\) and \((-1, -1)\) do not form a basis for \({\mathbb R}^2\) since these two vectors lie along the same line.

If \(2 \times 2\) matrices and the rest of what we have described above make you nervous, you should work through the exercises at the end of this section.

Subsection3.1.3Finding Eigenvalues and Eigenvectors

A nonzero vector \({\mathbf v}\) is an eigenvector of \(A\) if \(A {\mathbf v} = \lambda {\mathbf v}\) for some \(\lambda \in {\mathbf R}\text{.}\) The constant \(\lambda\) is called an eigenvalue of \(A\text{.}\) Letting \begin{equation*} A = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \quad \text{and} \quad \mathbf v = \begin{pmatrix} x \\ y \end{pmatrix} \neq \mathbf 0, \end{equation*} we have \(A \mathbf x = \lambda \mathbf x\) or \(A \mathbf x - \lambda \mathbf x = \mathbf 0\text{.}\) In matrix form this is \begin{align*} \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} - \lambda \begin{pmatrix} x \\ y \end{pmatrix} \amp = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} - \begin{pmatrix} \lambda & 0 \\ 0 & \lambda \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}\\ \amp = \begin{pmatrix} a- \lambda & b \\ c & d - \lambda \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix}\\ \amp = \begin{pmatrix} 0 \\ 0 \end{pmatrix}. \end{align*} This matrix equation is certainly true if \((x, y) = (0, 0)\text{.}\) However, we seek nonzero solutions to this system. This will occur exactly when the determinant of \begin{equation*} A - \lambda I = \begin{pmatrix} a- \lambda & b \\ c & d - \lambda \end{pmatrix} \end{equation*} is zero. In this case \begin{equation*} \det(A - \lambda I) = \det\begin{pmatrix} a - \lambda & b \\ c & d - \lambda \end{pmatrix} = \lambda^2 - (a + d) \lambda + (ad - bc). \end{equation*} We say that \begin{equation*} \det(A - \lambda I) = \lambda^2 - (a + d) \lambda + (ad - bc) \end{equation*} is the characteristic polynomial of \(A\text{.}\) We summarize the results of this discussion in the following theorem.

Example3.6

Suppose that we wish to find the eigenvalues and associated eigenvectors of \begin{equation*} A = \begin{pmatrix} 1 & 2 \\ 4 & 3 \end{pmatrix}. \end{equation*} To find the eigenvalues and eigenvectors for \(A\text{,}\) we must solve the equation \begin{equation*} A \begin{pmatrix} x \\ y \end{pmatrix} = \lambda \begin{pmatrix} x \\ y \end{pmatrix}. \end{equation*} If we let \(I\) denote the \(2 \times 2\) identity matrix, \begin{equation*} I = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \end{equation*} we can rewrite this equation in the form \begin{equation} (A - \lambda I) \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}. \label{equation-linear01-characteristic}\tag{3.2} \end{equation} We know that \(A - \lambda I\) is a \(2 \times 2\) matrix and that this system will only have nonzero solutions if \(\det(A - \lambda I) = 0\text{.}\) In our example, \begin{align*} \det(A - \lambda I) & = \det\begin{pmatrix} 1 - \lambda & 2 \\ 4 & 3 - \lambda \end{pmatrix} \\ & = (1 - \lambda) (3 - \lambda ) - 8\\ & = \lambda^2 - 4\lambda - 5\\ & = (\lambda - 5)(\lambda +1 ). \end{align*} Thus, \(\lambda = 5\) or \(-1\text{.}\)

To see this from a different perspective, we will rewrite equation (3.2) as \begin{align*} x + 2 y & = \lambda x\\ 4 x + 3 y & = \lambda y. \end{align*} This system is equivalent to \begin{align*} (1 - \lambda) x + 2 y & = 0\\ 4 x + (3 - \lambda) y & = 0 \end{align*} which can be reduced to \begin{align*} (1 - \lambda) x + 2 y & = 0\\ (\lambda^2 - 4\lambda - 5) y & = 0. \end{align*} Therefore, either \(\lambda = 5\) or \(\lambda = -1\) to obtain a nonzero solution.

  • If \(\lambda = 5\text{,}\) the first equation in the system becomes \(-2x + y = 0\text{,}\) and the eigenvectors corresponding to this eigenvalue are the nonzero solutions of this equation. That is, a vector must be a nonzero multiple of \((1, 2)\) to be an eigenvector of \(A\) corresponding to \(\lambda = 5\text{.}\)
  • If \(\lambda = -1\text{,}\) then the corresponding eigenvectors are the nonzero multiples of \((1, -1)\text{.}\)

Subsection3.1.4Important Lessons

  • The roots of the characteristic polynomial, \(\det(A - \lambda I)\text{,}\) of a matrix \(A\) are the eigenvalues of \(A\text{.}\) Given a specific eigenvalue, \(\lambda\text{,}\) for a matrix \(A\text{,}\) the eigenvectors associated with \(A\) are the nonzero solutions of the system of equations \begin{equation*} (A - \lambda I) \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 0 \\ 0 \end{pmatrix}. \end{equation*}
  • If \({\mathbf v}_1\) and \({\mathbf v}_2\) are eigenvectors of two distinct real eigenvalues of a matrix \(A\text{,}\) then \({\mathbf v}_1\) and \({\mathbf v}_2\) are linearly independent.

Subsection3.1.5Exercises

1

If \begin{equation*} A = \begin{pmatrix} 1 & -2 \\ 3 & 1 \end{pmatrix} \quad \text{and} \quad B = \begin{pmatrix} 4 & 1 \\ -1 & -2 \end{pmatrix}, \end{equation*} find each of the following.

  1. \(A + B\)
  2. \(2A - 3B\)
  3. \(AB\)
  4. \(BA\)
  5. \(A^{-1}\)
  6. \(B^{-1}\)
2

If \begin{equation*} A = \begin{pmatrix} 2 & 1 - i \\ 2 - i & 2 \end{pmatrix} \quad \text{and} \quad B = \begin{pmatrix} 4i & 1 - i \\ 1 + 3i & -2 -i \end{pmatrix}, \end{equation*} find each of the following.

  1. \(A + B\)
  2. \(3A - 2B\)
  3. \(AB\)
  4. \(BA\)
3

If \begin{equation*} A = \begin{pmatrix} 3 & -2 \\ 0 & -1 \end{pmatrix}, \mathbf x = \begin{pmatrix} 4 \\ 1 \end{pmatrix}, \quad \text{and} \quad \mathbf y = \begin{pmatrix} -2 \\ 3 \end{pmatrix}, \end{equation*} find each of the following.

  1. \(A \mathbf x\)
  2. \(A \mathbf y\)
  3. \(\mathbf x^T \mathbf y\)
  4. \(\mathbf y^T \mathbf x\)
4

Let \({\mathbf x} = (x_1, x_2)\) and \({\mathbf y} = (y_1, y_2)\text{.}\) Prove that \({\mathbf x}\) and \({\mathbf y}\) are linearly independent if and only if \begin{equation*} \det \begin{pmatrix} x_1 & y_1 \\ x_2 & y_2 \end{pmatrix} \neq 0. \end{equation*}

Subsection3.1.6Project—Finding Eigenvalues and Eigenvectors

Sage can be used to find eigenvalues and eigenvectors for a matrix \(A\) for now. Consider the matrix \begin{equation*} A = \begin{pmatrix} 1 & 3 \\ 1 & -1 \end{pmatrix}. \end{equation*} Using Sage, we would enter the matrix \(A\) as follows.

We can use the following command to find the eigenvalues of \(A\text{.}\)

Sage will also allow us find eigenvectors for each of the eigenvalues of \(A\text{.}\)

Thus, the matrix \(A\) has two eigenvalues: \(\lambda_1 = 2\) with eigenvector \(\mathbf v_1 = (1, 1/3)\) and \(\lambda_2 = -2\) with eigenvector \(\mathbf v_2 = (1, -1)\text{.}\)

There is a third entry in the Sage output which refers to the multiplicity of the eigenvalue. In the previous example, the multiplicity is 1. In the matrix \begin{equation*} B = \begin{pmatrix} 1 & 1 \\ -1 & 3 \end{pmatrix}. \end{equation*} in the Sage cell below, we obtain a single repeated eigenvalue \(\lambda = 2\) and only one eigenvector \(\mathbf v = (1,1)\text{.}\) The multiplicity of this eigenvalue is 2. In our previous examples, we obtained two linearly independent eigenvalues allowing us to solve initial value problems given a general solution.

We may also matrices such as \begin{equation*} C = \begin{pmatrix} 4 & 1 \\ -1 & 4 \end{pmatrix} \end{equation*} has complex eigenvalues, \(\lambda = 4 - i\) and \(\mu = 4 + i\text{.}\) Eigenvectors for \(\lambda\) and \(\mu\) are \(\mathbf u = (1, -i)\) and \(\mathbf v = (1, i)\text{,}\) respectively.