WordGPT
← Math: from algebra to machine learning

Linear algebra · Part 4 of 5

Eigenvalues and eigenvectors

Eigenvectors are the directions a matrix only stretches, and eigenvalues are the stretch factors; find them from the characteristic polynomial and see why they make repeated multiplication easy.

Linear algebra9 min read

Multiply a vector by a matrix and it usually changes direction as well as length. But for most square matrices there are a few special directions that come out pointing the same way (or exactly backwards), only longer or shorter. Those directions are called eigenvectors, and the stretch factors are the eigenvalues.

They matter because they turn a tangle into something simple. Applying a matrix ten times to a vector in one of these directions is just multiplying by a number ten times. Principal component analysis, vibration modes of a bridge, and the long-run behaviour of many systems all come down to finding eigenvalues and eigenvectors. You will learn to find them for matrices by hand and to see when they do not exist as real numbers.

What an eigenvector is

A nonzero vector is an eigenvector of a square matrix if multiplying by only scales it:

The number is the matching eigenvalue. If the vector is stretched, if it is shrunk, and if it is flipped to point the opposite way. If the vector is squashed onto the zero vector, which is allowed: the eigenvalue may be zero, but the eigenvector itself must never be the zero vector.

An eigenvector is really a direction. If works, so does every nonzero multiple of it, such as or , with the same eigenvalue.

An arrow v of length 1 and the arrow Av, with a dotted line through v. For the matrix with rows (4, 1) and (2, 3), Av lies on that line only when v points along (1, 1), where Av is 5 times v, or along (1, -2), where Av is 2 times v.vAv
Turn v with the slider and see Av, what the matrix makes of it. Most of the time Av points somewhere else. In two special directions it stays on the same line as v, only longer or shorter; tick the box to see them.

Example 1. Checking an eigenvector Show that is an eigenvector of and find its eigenvalue.

Multiply by .

The result is times the original vector.

So is an eigenvector of with eigenvalue .

Finding the eigenvalues

To find eigenvalues, rewrite as , where is the identity matrix. We want a nonzero solution . A nonzero solution exists exactly when has no inverse, that is, when its determinant is zero:

This is the characteristic equation. Notice that is subtracted only from the diagonal entries, because that is what does. For a matrix it expands to

where the trace is the sum of the diagonal entries.

Parabola λ² − 7λ + 10 from 0 to 7, crossing the horizontal axis at λ = 2 and λ = 5.01234567510λ = 2λ = 5λ² − 7λ + 10
The characteristic polynomial of A crosses zero exactly at the eigenvalues λ = 2 and λ = 5.

Example 2. Eigenvalues of a 2x2 matrix Find the eigenvalues of .

The trace is and the determinant is . The characteristic equation is

Factor it.

So the eigenvalues are and .

Finding the eigenvectors

Once you have an eigenvalue , put it back into and solve for . Because is singular, its two rows are not independent. Unless the matrix is all zeros, they describe the same condition, so you get one equation in two unknowns and a whole line of solutions. Pick any convenient nonzero point on that line.

A tidy way to check all your work at once: put the eigenvectors as the columns of a matrix and the eigenvalues on the diagonal of . Then holds exactly when every column really is an eigenvector for its eigenvalue.

Plane with segments from the origin along (1, 1) to (5, 5) and along (1, −2) to (2, −4), showing each eigenvector stretched along its own line by 5 and 2.1.417.072.244.47OV1AV1V2AV2
Both eigenvectors of A stay on their own lines: (1, 1) goes to 5·(1, 1) and (1, −2) goes to 2·(1, −2).
Interactive 2x2 matrix with four sliders a, b, c, d. The unit square becomes a parallelogram; for the starting matrix with rows (4, 1) and (2, 3) the arrow (1, 1) is sent to (5, 5).det = 10
The sliders are the four entries of the matrix. The blue shape is where the matrix sends a 1 by 1 square, and the red and green arrows are where (1, 0) and (0, 1) land. With the starting matrix, the arrow (1, 1) comes out as (5, 5): the same direction, 5 times longer.

Example 3. Eigenvectors of a 2x2 matrix Find an eigenvector for each eigenvalue of and check them.

For , solve .

Both rows say , so take .

For , solve .

Both rows say , so . Take .

Check with and :

Both products agree, so both eigenvectors are correct.

Shortcuts and useful facts

A few facts save a lot of work.

  • The sum of the eigenvalues equals the trace, and their product equals the determinant. For the matrix above, and .
  • For a triangular matrix (all zeros above or all zeros below the diagonal) the eigenvalues are just the diagonal entries.
  • An eigenvalue of appears exactly when , that is, when is not invertible.
  • For a symmetric real matrix () the eigenvalues are always real, and eigenvectors that belong to different eigenvalues are perpendicular.

Example 4. A triangular matrix Find the eigenvalues of .

The matrix is upper triangular, so the eigenvalues are the diagonal entries. To see why, expand the characteristic equation.

The eigenvalues are and .

When there are fewer directions than you expect

Two things can go wrong for a matrix.

No real eigenvalues. A rotation by turns every direction, so no direction survives. The matrix rotates the plane by . Its characteristic equation is , which has no real solution. (It has two complex eigenvalues, and , which you will meet later.)

A repeated eigenvalue with only one direction. The matrix below has the eigenvalue twice, yet only a single line of eigenvectors.

Example 5. One eigenvector direction only For , find the eigenvalues and show that there is only one independent eigenvector.

The matrix is triangular, so the eigenvalue is the diagonal entry , repeated twice. Now solve .

This forces and leaves free, so every eigenvector is a multiple of , the -axis direction. The matrix has rank , which leaves a single free direction.

Why eigenvectors make powers easy

If is an eigenvector then , and applying again gives , then . No matrix multiplication is needed.

A vector that is not an eigenvector can often be written as a combination of eigenvectors. Then each piece just gets multiplied by its own power of its eigenvalue. The piece with the largest eventually dominates, which is why the largest eigenvalue decides the long-run behaviour of repeated multiplication.

Growth of the two pieces 2·λⁿ and 2ⁿ over n = 0 to 3, with a slider for the larger eigenvalue λ, showing the λⁿ piece dominating as n grows.00.511.522.530501001502002502·λⁿ piece2ⁿ piece
Each piece of the vector grows like its own eigenvalue to the power n; drag λ and see how the larger eigenvalue quickly dominates the long-run behaviour.

Example 6. A matrix applied twice With and , compute using the eigenvectors for and for .

Write as a combination of the eigenvectors.

This gives and , so and . Now apply twice, scaling each piece by .

Check directly: and .

Common mistakes

  • Accepting as an eigenvector: is true for every , which is why eigenvectors are required to be nonzero.
  • Computing by subtracting from every entry; subtract only from the diagonal, that is, form .
  • Thinking means there is no eigenvector; it means has a nonzero solution, so and is not invertible.
  • Counting and as two different eigenvectors; they lie along the same direction, and one direction is one eigenvector up to scaling.
  • Assuming every real matrix has real eigenvalues; a rotation has none, because has no real solution.

Practice

  1. Find the eigenvalues of .
  2. Find the eigenvalues of .
  3. One eigenvalue of is . Use the fact that the product of the eigenvalues equals the determinant to find the other eigenvalue.
  4. Find the eigenvalues of the triangular matrix .
  5. A matrix has trace and determinant . Find its eigenvalues.
Answers
  1. and , from .
  2. and , from .
  3. , because and .
  4. , the diagonal entries of a triangular matrix.
  5. and , from .

Next, Singular value decomposition extends these ideas to matrices that are not square.

Part of the free WordGPT Learn section. WordGPT is an AI writing workspace for documents, resumes, websites and study.

    Eigenvalues and eigenvectors — Math: from algebra to machine learning | WordGPT