Skip to main content
\(\newcommand{\trace}{\operatorname{tr}} \newcommand{\real}{\operatorname{Re}} \newcommand{\imaginary}{\operatorname{Im}} \newcommand{\lt}{<} \newcommand{\gt}{>} \newcommand{\amp}{&} \)

Section3.9The Matrix Exponential

Consider the linear system \begin{align*} x' & = -17x -46y -7z\\ y' & = 8x + 21y + 3z\\ z' & = -7x - 15y - z \end{align*} The matrix associated with this system is \begin{equation*} A = \begin{pmatrix} -17 & -46 & -7 \\ 8 & 21 & 3 \\ -7 & -15 & -1 \end{pmatrix}. \end{equation*} The characteristic polynomial of \(A\) is \begin{equation*} p(\lambda) = \det(A - \lambda I) = \lambda^3 - 3\lambda^2 + 3\lambda - 1 = (\lambda - 1)^3 \end{equation*} hence, there is only a single eigenvalue \(\lambda = 1\text{.}\) Moreover, we can only find a single linearly independent eigenvector \((1, -1, 4)\text{.}\) Thus, \begin{equation*} {\mathbf x}(t) = c e^{-t} \begin{pmatrix} 1 \\ -1 \\ 4\end{pmatrix} \end{equation*} is a solution to our system. However, this is not the general solution to the system. We can only solve initial value problems where the initial condition lies on the line through the origin containing the vector \((1, -1, 4)\text{.}\) To construction a general solution to our system, we will need two other linearly independent solutions.

Subsection3.9.1The Exponential of a Matrix

Our goal is to construct a solution to the initial value problem \begin{align*} {\mathbf x}' & = A {\mathbf x}\\ {\mathbf x}(0) & = {\mathbf x}_0, \end{align*} where \(A\) is an \(n \times n\) matrix. Recalling that the solution to the initial value problem \begin{align*} x' & = kx\\ x(0) & = x_0 \end{align*} is \(x(t) = x_0 e^{kt}\text{,}\) we might guess that a solution to the initial value problem \begin{align*} {\mathbf x}' & = A {\mathbf x}\\ {\mathbf x}(0) & = {\mathbf x}_0 \end{align*} has the form \begin{equation*} {\mathbf x}(t) = e^{tA} {\mathbf x}_0 \end{equation*} if we can make sense of the expression \(e^{tA}\text{.}\)

We will define the exponential of \(A\) using the power series for \(e^t\text{.}\) Thus, \begin{equation*} e^A = I + A + \frac{1}{2!} A^2 + \frac{1}{3!} A^3 + \cdots = \sum_{k=0}^\infty \frac{1}{k!} A^k, \end{equation*} where \(A\) is an \(n \times n\) matrix, where \(A^0 = I\) by convention. Each term makes sense in our definition since each is an expression of \(n \times n\) matrices; however, there are some issues surrounding the convergence of the power series. A series, even a series whose individual terms are matrix expressions, converges if and only if its partial sums, \begin{equation*} S_N = \sum_{k=0}^N \frac{1}{k!} A^k \end{equation*} converge. Although we shall not provide a proof, the matrix exponential \(e^A\) converges for all \(A\text{.}\)

Example3.71

Let us compute the exponential of \begin{equation*} A = \begin{pmatrix} s & 0 \\ 0 & t \end{pmatrix}. \end{equation*} Actually, this is quite easy, \begin{align*} e^A & = I + A + \frac{1}{2!} A^2 + \frac{1}{3!} A^3 + \cdots\\ & = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} + \begin{pmatrix} s & 0 \\ 0 & t \end{pmatrix} + \frac{1}{2!} \begin{pmatrix} s^2 & 0 \\ 0 & t^2 \end{pmatrix} + \frac{1}{3!} \begin{pmatrix} s^3 & 0 \\ 0 & t^3 \end{pmatrix} + \cdots\\ & = \begin{pmatrix} 1 + s + s^2/2! + s^3/3! + \cdots & 0 \\ 0 & 1 + t + t^2/2! + t^3/3! + \cdots \end{pmatrix}\\ & = \begin{pmatrix} e^s & 0 \\ 0 & e^t \end{pmatrix}. \end{align*}

Thus, solving linear systems is simply a matter of computing matrix exponentials. The problem is that matrix exponentials may not be so easy to compute.

Example3.74

The matrix \begin{equation*} A = \begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix} \end{equation*} has repeated eigenvalues \(\lambda = -2\text{.}\) If we try to compute \(e^{tA}\text{,}\) then \begin{align*} e^{tA} & = I + tA + \frac{t^2}{2!} A^2 + \frac{t^3}{3!} A^3 + \cdots\\ & = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} + t\begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix} + \frac{t^2}{2!} \begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix}^2 + \frac{t^3}{3!} \begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix}^3 + \cdots\\ & = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} + t\begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix} + \frac{t^2}{2!} \begin{pmatrix} -20 \amp -16 \\ -36 \amp 28 \end{pmatrix} + \frac{t^3}{3!} \begin{pmatrix} 64 \amp 48 \\ -108 \amp -80 \end{pmatrix} + \cdots\\ & = \begin{pmatrix} 1 + t - 20t^2/2! + 64t^3/3! + \cdots & 4t - 16t^2/2! + 48t^3/ 3! + \cdots \\ -9t - 36t^2/2! - 108t^3/3! + \cdots & 1 - 8t + 28t^2/2! - 80t^3/3! + \cdots \end{pmatrix}. \end{align*} It is not at all clear that this series converges to a matrix whose entries can be expressed in terms of elementary functions.

Now let us see how we can use the matrix exponential to solve a linear system as well as invent a more direct way to compute the matrix exponential.

The matrix exponential shares several properties with the function exponential function \(e^x\) that we studied in calculus.

Example3.77

Now let us compute \(e^{tA}\) once again for \begin{equation*} A = \begin{pmatrix} 4 \amp 4 \\ -9 \amp -8 \end{pmatrix}. \end{equation*} First notice that \begin{equation*} A = \lambda I + (A - \lambda I). \end{equation*} Since the identity matrix \(I\) commutes with every matrix, we know that \begin{equation*} e^{t A} = e^{t(\lambda I + (A - \lambda I))} = e^{t\lambda I} e^{t(A - \lambda I)}. \end{equation*} We also know that \(e^{\lambda t I } = e^{\lambda t} I\) by Example 3.71. Thus, \begin{equation*} e^{ t A} = e^{t(\lambda I + (A - \lambda I))} = e^{t\lambda I} e^{t (A - \lambda I)} = e^{\lambda t} e^{t (A - \lambda I)}. \end{equation*} The matrix \(A\) has repeated eigenvalue \(\lambda = -2\text{.}\) Consequently, \begin{equation*} A - \lambda I = A + 2I = \begin{pmatrix} 6 & 4 \\ -9 & -6 \end{pmatrix}, \end{equation*} and \((A - \lambda I)^2\) is the zero matrix. Thus, \begin{align*} e^{ t A} & = e^{\lambda t} e^{t (A - \lambda I)}\\ & = e^{\lambda t} \left( I + t (A - \lambda I ) + \frac{t^2}{2!} (A - \lambda I )^2 + \frac{t^2}{3!} (A - \lambda I )^3 + \cdots \right)\\ & =e^{- 2t} \left( I + t (A + 2I ) + \frac{t^2}{2!} (A + 2I )^2 + \frac{t^2}{3!} (A + 2I )^3 + \cdots \right)\\ & =e^{- 2t} \left( I + t (A + 2I ) \right)\\ & = e^{-2t} \begin{pmatrix} 1 + 6t \amp 4t \\ -9t \amp -6t \end{pmatrix}. \end{align*}

Our example suggests at the following proposition. We leave the proof of this proposition as an exercise.

Example3.79

We are now ready return to our original system \({\mathbf x}' = A {\mathbf x}\text{,}\) where \begin{equation*} A = \begin{pmatrix} -17 & -46 & -7 \\ 8 & 21 & 3 \\ -7 & -15 & -1 \end{pmatrix}. \end{equation*} This matrix has a single eigenvalue \(\lambda = 1\text{.}\) It is easy to show that the only nonzero powers of \(A - \lambda I = A - I\) are \begin{align*} (A + I) & = \begin{pmatrix} -18 & -46 & -7 \\ 8 & 20 & 3 \\ -7 & -15 & -2 \end{pmatrix}\\ (A + I)^2 & = \begin{pmatrix} 5 & 13 & 2 \\ -5 & -13 & -2 \\ 20 & 52 & 8 \end{pmatrix}. \end{align*} Therefore, \begin{align*} e^{tA} & = e^{t} \left( I + t (A+I ) + \frac{t^2}{2!} (A + I )^2 \right)\\ & = e^{t} \begin{pmatrix} 1 - 18t + 5t^2/2 \amp -46t +13t^2/2 \amp - 7t + t^2 \\ 8t - 5t^2/2 \amp 1 + 20t -13t^2/2 \amp 3t - t^2 \\ -7t + 10t^2 \amp -15t + 26t^2 \amp 1 - 2t + 4t^2 \end{pmatrix}. \end{align*} Now, to compute three linearly independent solutions for \({\mathbf x}' = A {\mathbf x}\text{,}\) we simply compute \(e^{t A} {\mathbf v}\) for three linearly independent vectors. We will use the standard basis vectors \begin{equation*} {\mathbf e}_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, {\mathbf e}_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}, {\mathbf e}_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}. \end{equation*} Thus, the general solution to our system is \begin{equation*} {\mathbf x}(t) = c_1 e^{-t} \begin{pmatrix} 1 - 18t + 5t^2/2 \\ 8t - 5t^2/2 \\ -7t + 10t^2 \end{pmatrix} + c_2 e^{-t} \begin{pmatrix} -46t +13t^2/2 \\ 1 + 20t -13t^2/2 \\ -15t + 26t^2 \end{pmatrix} + c_3 e^{-t} \begin{pmatrix} - 7t + t^2 \\ 3t - t^2 \\ 1 - 2t + 4t^2 \end{pmatrix}. \end{equation*}

Subsection3.9.2Generalized Eigenvalues

Example3.80

Consider the system \({\mathbf x}' = A {\mathbf x}\text{,}\) where \begin{equation*} A = \begin{pmatrix} 7 \amp -1 \amp 2 \\ 18 \amp -2 \amp 6 \\ -9 \amp 2 \amp -1 \end{pmatrix}\text{.} \end{equation*} The characteristic polynomial of \(A\) is \begin{equation*} \det(A - \lambda I) = \lambda^3 - 4\lambda^2 + 5 \lambda - 2 = (\lambda - 1)^2 (\lambda - 2). \end{equation*} The eigenvalue \(\lambda_1 = 1\) has eigenvector \(\mathbf v_1 = (2, 6, -3)\) and the eigenvalue \(\lambda_2 = 2\) has eigenvector \(\mathbf v_2 = (1, 3, -1)\text{.}\) Thus, we can find two linearly independent solutions in this case \begin{equation*} {\mathbf x}_1(t) = e^{t} \begin{pmatrix} 2 \\ 6 \\ -3 \end{pmatrix} \qquad\text{and}\qquad {\mathbf x}_2(t) = e^{2t} \begin{pmatrix} 1 \\ 3 \\ -1 \end{pmatrix}. \end{equation*} Since \(\lambda_1 = 1\) has multiplicity two and we can find only one linearly independent eigenvector, it is not possible to apply Proposition 3.78 in this case.

If we consider the exponential \begin{align*} e^{ t A} {\mathbf v} & = e^{\lambda_1 t} e^{t (A - \lambda_1 I)} {\mathbf v}\\ & = e^{\lambda_1 t} \left( {\mathbf v} + t (A - \lambda_1 I ){\mathbf v} + \frac{t^2}{2!} (A - \lambda_1 I )^2 {\mathbf v} + \frac{t^2}{3!} (A - \lambda_1 I )^3 {\mathbf v} + \cdots \right) \end{align*} where \(\mathbf v, \mathbf v_1\) and \(\mathbf v_2\) are linearly independent, our goal is to choose \({\mathbf v}\) for which the series truncates. That is, we must look for vectors \({\mathbf v}\) such that \((A - \lambda_1 I)^k {\mathbf v} = (A - I)^k{\mathbf v} = {\mathbf 0}\text{.}\) If \(k = 1\text{,}\) then \((A - \lambda_1 I) {\mathbf v} = (A - I){\mathbf v} = {\mathbf 0}\text{,}\) which means that \({\mathbf v}\) is an eigenvector. Thus, \({\mathbf v}\) must be a multiple of \(\mathbf v_1 = (2, 6, -3)\) in this case. Since we already know that the eigenspace associated with this eigenvector has dimension one and is generated by \(\mathbf v_1\text{,}\) we must consider higher powers.

Since \begin{equation*} (A - \lambda I) = (A - I) = \begin{pmatrix} 6 \amp -1 \amp 2 \\ 18 \amp -3 \amp 6 \\ -9 \amp 2 \amp -2 \end{pmatrix}, \end{equation*} we have \begin{equation*} (A - \lambda I)^2 = (A - I)^2 = \begin{pmatrix} 0 & 1 & 2 \\ 0 & 3 & 6 \\ 0 & -1 & -2 \end{pmatrix}. \end{equation*} The nullspace of this matrix has dimension two. Certainly, \(\mathbf v_1 = (2, 6, -3)\) is in the nullspace of \((A - I)^2\text{,}\) since it is the nullspace of \(A - I\text{.}\) We wish to find a vector in the nullspace that is not a multiple of the vector \(\mathbf v_1\) that is also in the nullspace of \(A - I\text{.}\) The vector \(\mathbf v = (0, 1, 0)\) will do. Now our series truncates, \begin{align*} {\mathbf x}_3(t) & = e^{ t A} {\mathbf v}\\ & = e^{\lambda_1 t} e^{t (A - \lambda_1 I)} {\mathbf v}\\ & = e^{\lambda_1 t} \left( {\mathbf v} + t (A - \lambda_1 I ){\mathbf v} \right)\\ & = e^{\lambda_1 t} \left[ \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} + t (A - I )\begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} \right]\\ & = e^{t} \begin{pmatrix} t \\ 1 + 3t \\ - t \end{pmatrix} \end{align*} We now have a general solution for our system, \begin{equation*} {\mathbf x}(t) = c_1 e^{t} \begin{pmatrix} 2 \\ 6 \\ -3 \end{pmatrix} + c_2 e^{2t} \begin{pmatrix} 1 \\ 3 \\ -1 \end{pmatrix} + c_3 e^{t} \begin{pmatrix} t \\ 1 + 3t \\ - t \end{pmatrix}. \end{equation*}

If \(\lambda\) is an eigenvalue of \(A\) and \((A - \lambda I)^p {\mathbf v} = {\mathbf 0}\) for some \(p \geq 1\text{,}\) then \({\mathbf v}\) is called a generalized eigenvector of \(A\text{.}\) When eigenvalues have algebraic multiplicity greater than one, we can compute extra solutions by looking for vectors in the nullspace of \((A - \lambda I)^p\) for \(p > 1\text{.}\) The following theorem tells us that this is always possible. We leave the proof of the theorem as an exercise in linear algebra.

We now have a procedure for finding \(q\) linearly independent solutions corresponding to an eigenvalue \(\lambda\) of multiplicity \(q\text{.}\)

  1. Find the smallest integer \(p\) such that the nullspace of \((A - \lambda I)^p\) has dimension \(q\text{.}\)
  2. Find a basis \(\{ {\mathbf v}_1, \ldots, {\mathbf v}_q \}\) for the nullspace of \((A - \lambda I)^p\text{.}\)
  3. For each \({\mathbf v}_j\) (\(j = 1, 2, \ldots, q\)), we have the solution \begin{align*} {\mathbf x}_j(t) & = e^{tA} {\mathbf v}_j\\ & = e^{\lambda t} \left( {\mathbf v}_j + t(A - \lambda I){\mathbf v}_j + \cdots + \frac{t^{p-1}}{(p-1)!}(A - \lambda I)^{p-1}{\mathbf v}_j. \right). \end{align*}

This procedure works for complex eigenvalues as well as real. If \(\lambda = \alpha + i \beta\) has eigenvector \({\mathbf z} = \mathbf x + i \mathbf y\text{,}\) then set \({\mathbf x} = \real{\mathbf z}\) and \({\mathbf y} = \imaginary{\mathbf z}\text{.}\)

Subsection3.9.3Important Lessons

  • If \(A\) is an \(n \times n\) matrix, we define the exponential of \(A\) to be \begin{equation*} e^A = I + A + \frac{1}{2!} A^2 + \frac{1}{3!} A^3 + \cdots = \sum_{k=0}^\infty \frac{1}{k!} A^k. \end{equation*}
  • If \(A\) is an \(n \times n\) matrix, then \begin{equation*} \frac{d}{dt} e^{tA} = A e^{tA}. \end{equation*}
  • Let \(A\) be an \(n \times n\) matrix. Then \({\mathbf x}(t) = e^{tA}{\mathbf x}_0\) is the solution to the initial value problem \begin{align*} {\mathbf x}' & = A {\mathbf x}\\ {\mathbf x}(0) & = {\mathbf x}_0 \end{align*}
  • Let \(\lambda\) be an eigenvalue of an \(n \times n\) matrix A. If \({\mathbf v}\) is an eigenvector for \(\lambda\text{,}\) then \(e^{tA} {\mathbf v} = e^{\lambda t} {\mathbf v}\text{.}\)
  • Let \(A\) and \(B\) be \(n \times n\) matrices. Then

    • \(Ae^A = e^A A\text{;}\)
    • if \(AB = BA\text{,}\) then \(e^A e^B = e^{A+B}\text{;}\)
    • \(e^A\) is nonsingular with inverse \(e^{-A}\text{.}\)
  • If \(A\) is an \(n \times n\) matrix with a single eigenvalue \(\lambda\text{,}\) then there exists nonnegative integer \(k \lt n\) such that \begin{equation*} e^{tA} = e^{\lambda t} \left( I + t(A - \lambda I) + \frac{t^2}{2!} (A - \lambda I)^2 + \cdots + \frac{t^k}{k!} (A - \lambda I)^k \right). \end{equation*}
  • Suppose that \(\lambda\) is an eigenvalue of \(A\) with multiplicity \(q\text{.}\) Then there exists an integer \(p \leq q\) such that the dimension of the nullspace of \((A - \lambda I)^p\) is \(q\text{.}\)
  • The procedure for finding \(q\) linearly independent solutions corresponding to an eigenvalue \(\lambda\) of multiplicity \(q\) is the following.

    • Find the smallest integer \(p\) such that the nullspace of \((A - \lambda I)^p\) has dimension \(q\text{.}\)
    • Find a basis \(\{ {\mathbf v}_1, \ldots, {\mathbf v}_q \}\) for the nullspace of \((A - \lambda I)^p\text{.}\)
    • For each \({\mathbf v}_j\) (\(j = 1, 2, \ldots, q\)), we have the solution \begin{align*} {\mathbf x}_j(t) & = e^{tA} {\mathbf v}_j\\ & = e^{\lambda t} \left( {\mathbf v}_j + t(A - \lambda I){\mathbf v}_j + \cdots + \frac{t^{p-1}}{(p-1)!}(A - \lambda I)^{p-1}{\mathbf v}_j. \right). \end{align*}

    The procedure works for complex eigenvalues as well as real. If \(\lambda = \alpha + i \beta\) has eigenvector \({\mathbf z}\text{,}\) then set \({\mathbf x} = \real{\mathbf z}\) and \({\mathbf y} = \imaginary{\mathbf z}\text{.}\)

Subsection3.9.4Exercises

1

Solve the system \begin{align*} x' & = -5x + 15y - 4z\\ y' & = -x + 5y - z\\ z' & =4 x -6y + 3z\\ x(0) & = -1\\ y(0) & = 2\\ z(0) & = 0 \end{align*}

2

Find the general solution of the system \begin{align*} x_1' & = 6x_1 + 9x_2 + x_3 + 4 x_4\\ x_2' & = -x_1 + x_2 + x_3\\ x_3' & = 3x_1 + 8x_2 + 4x_3 + 4 x_4\\ x_4' & = -3x_1 - 8x_2 -7 x_3 - 5 x_4 \end{align*}

3

Let \(a x'' + b x' + cx = 0\text{,}\) where \(a \neq 0\) and \(b^2 - 4ac = 0\text{.}\)

  1. Show that \(x_1(t) = e^{-bt/2a}\) is a solution to \(a x'' + b x' + cx = 0\text{.}\)
  2. Assume that \begin{equation*} y = v(t) x_1(t) = v(t) e^{-bt/2a} \end{equation*} is a solution to \(a x'' + b x' + cx = 0\) and show that \(v(t) = c_1 + c_2 t\text{.}\) Thus, \begin{equation*} x(t) = c_1 e^{-bt/2a} + c_2 t e^{-bt/2a} \end{equation*} is a general solution for \(a x'' + b x' + cx = 0\text{.}\)
4

Consider the equation \begin{equation*} y'' - (2 \alpha -1) y' + \alpha (\alpha - 1) y =0. \end{equation*} Determine all values of \(\alpha\text{,}\) if any, for which all solutions tend toward zero as \(t \to \infty\text{.}\) Also, determine the values of \(\alpha\text{,}\) if any, for which all nonzero solutions become unbounded as \(t \to \infty\text{.}\)

5

Solve each of the following initial value problems.

  1. \begin{align*} y'' - 2y' + 5y & = 0\\ y(\pi/2) & = 0\\ y'(\pi/2) & = 2 \end{align*}
  2. \begin{align*} 9y'' - 12y' + 4y & = 0\\ y(0) & = 2\\ y'(0) & = -1 \end{align*}
  3. \begin{align*} y'' + 8y' -9y & = 0\\ y(1) & = 1\\ y'(1) & = 0 \end{align*}
  4. \begin{align*} y'' + 2ay' + (a^2 + 1)y & = 0\\ y(0) & = 1\\ y'(0) & = 0 \end{align*}
6

Reduction of Order. Suppose that \(x_1(t)\) is a solution (not identically zero) to the equation \begin{equation*} x'' + p(t) x' + q(t) x = 0. \end{equation*}

  1. Assume that \(x(t) = v(t) x_1(t)\) is a solution to \(x'' + p(t) x' + q(t) x = 0\) and derive the equation \begin{equation} x_1 v'' +(2x_1' + px_1)v' = 0.\label{equation-exercise-linear08-reduction-of-order-1}\tag{3.16} \end{equation}
  2. Let \(u = v'\) and show that ((3.14)) is a first-order linear differential equation in \(u\text{.}\)
  3. Show that \(x_1(t) = 1/t\) is a solution to \begin{equation} 2 t^2 x'' + 3t x' - x = 0\label{equation-exercise-linear08-reduction-of-order-2}\tag{3.17} \end{equation} for \(t \gt 0\) and find a second linearly independent solution to ((3.15)).
7

Euler Equations. An important class of second-order linear differential equations are equations of the form \begin{equation*} t^2 y'' + \alpha t y' + \beta y = 0, \end{equation*} where \(t \gt 0\) and \(\alpha\) and \(\beta\) are real constants. An equation of this form is called an Euler equation.

  1. Show that the substitution \(x = \ln t\) transforms an Euler equation into an equation of constant coefficients. {\em Hint}: Show that \begin{align*} \frac{dy}{dt} & = \frac{dx}{dt} \frac{dy}{dx}\\ \frac{d^2y}{dt^2} & = \left( \frac{dx}{dt} \right)^2 \frac{d^2y}{dx^2} + \frac{d^2x}{dt^2} \frac{dy}{dx}. \end{align*}
  2. Solve the equation \begin{equation*} t^2 y'' + 4ty' + 2y = 0. \end{equation*}

Subsection3.9.5Project