9. Eigenvalues and Eigenvectors

Have you ever heard the words eigenvalue and eigenvector? They are derived from the German word "eigen" which means "proper" or "characteristic." An eigenvalue of a square matrix is a scalar that is usually represented by the Greek letter (pronounced lambda). As you might suspect, an eigenvector is a vector. Moreover, we require that an eigenvector be a non-zero vector, in other words, an eigenvector can not be the zero vector. We will denote an eigenvector by the small letter x. All eigenvalues and eigenvectors satisfy the equation for a given square matrix, A.

Remark 27 Remember that, in general, the word scalar is not restricted to real numbers. We are only using real numbers as scalars in this book, but eigenvalues are often complex numbers.

Definition 9.1 Consider the square matrix A . We say that is an eigenvalue of A if there exists a non-zero vector x such that . In this case, x is called an eigenvector (corresponding to ), and the pair (,x) is called an eigenpair for A.

Let's look at an example of an eigenvalue and eigenvector. If you were asked if is an eigenvector corresponding to the eigenvalue for , you could find out by substituting x, and A into the equation

Now that you have seen an eigenvalue and an eigenvector, let's talk a little more about them. Why did we require that an eigenvector not be zero? If the eigenvector was zero, the equation would yield 0 = 0. Since this equation is always true, it is not an interesting case. Therefore, we define an eigenvector to be a non-zero vector that satisfies However, as we showed in the previous example, an eigenvalue can be zero without causing a problem. We usually say that x is an eigenvector corresponding to the eigenvalue if they satisfy Since each eigenvector is associated with an eigenvalue, we often refer to an x and that correspond to one another as an eigenpair. Did you notice that we called x "an" eigenvector rather than "the" eigenvector corresponding to ? This is because any non-zero, scalar multiple of an eigenvector is also an eigenvector. If you let c represent a scalar, then we can prove this fact through the following steps.

Since any non-zero, scalar multiple of an eigenvector is also an eigenvector, and are also eigenvectors corresponding to when

You have already computed eigenvectors in this course. When we studied Markov chains, you computed an eigenvector corresponding to A^T when you found the matrix to which the probabilities seemed to converge after many steps. Any row of that matrix is an eigenvector for A^T because all the rows of that matrix are the same. We write that row as a column vector when we use it as an eigenvector. The eigenvector that you found is called the dominant eigenvector.

Definition 9.2 The dominant eigenvector of a matrix is an eigenvector corresponding to the eigenvalue of largest magnitude (for real numbers, largest absolute value) of that matrix.

Although we only found one eigenvector, we found a very important eigenvector. Many of the "real world" applications are primarily interested in the dominant eigenpair. The method that you used to find this eigenvector is called the power method. The power method will be explained later in this chapter. An eigenvector corresponding to the transpose of a transition matrix is the transpose of any row of the matrix that A^k converges to as k grows, these rows are all the same. The dominant eigenvalue is always 1 for a transition matrix. Let's look at the example that we used in the Markov chain chapter. Consider the matrix

If k is large,

Therefore, an eigenvector corresponding to the dominant eigenvalue, 1, is Let's see if holds true.

Yes, the equation holds, so we have found an eigenpair corresponding to the transpose of the transition matrix.

Remark 28 For a transition matrix, the dominant eigenvalue is always 1. An eigenvector corresponding to 1 for A^T is the transpose of any row of A^* where A^* is the matrix to which A^k converges as k grows. The matrix for which we are finding an eigenpair must have been set up so that the columns (not the rows) add to 1 for the eigenvector to be read from A^* ; this is why we are dealing with A^T instead of A. These rules will require some modification if we are not dealing with a transition matrix.

If we do not have a transition matrix, can we still use the power method? Yes we can, but we need to modify the steps a bit because the dominant eigenvalue will not necessarily be the number one. Let us explain how to use the power method. An example follows the remarks to help clarify these steps.

1. Choose a vector and call it x₀. Set i = 0.

2. Multiply to get the next approximation for x using the formula x_i+1 = Ax_i.

3. Divide every term in x_i+1 by the last element of the vector and call the new vector .
4. Repeat steps 2 and 3 until and agree to the desired number of digits.
5. The vector obtained in step 4 is an approximate eigenvector corresponding to the dominant eigenvalue. We will call it x.

6. An approximation to the dominant eigenvalue is This is called the Rayleigh quotient of x.

Remark 29 Because any constant multiple of an eigenvector is an eigenvector, we did not have to divide by the last element in the vector in step 3. We could have divided by any element or not divided at all. We divided so that our vector would not grow too large and we could tell when we had converged. We divided by the last element of the vector so that we would have a well-defined algorithm for using the power method. The choice of the last element over any of the others was arbitrary. Therefore, if the last element is zero, divide by another element of the vector for that entire problem. When people program the power method on a computer, they usually divide by

,

which is defined as the length of the vector, so that they don't have to worry about whether or not an element is zero.

Remark 30 Some calculators will not let you divide a vector by a constant. On those calculators, you can multiply by the multiplicative inverse (reciprocal) of the constant.

Remark 31 You will probably not be able to directly input the Rayleigh quotient into your calculator. It will consider the numerator and denominator as 1 by 1 matrices. We consider 1 by 1 matrices to be the same as real numbers, but your calculator may not consider them the same. Since you cannot divide matrices, your calculator will probably give you an error message.

Therefore, it looks like an eigenvector is The corresponding eigenvalue is:

Let's try to find the dominant eigenpair of another matrix. Consider the matrix Again, we will choose but we could have chosen any vector of dimension 2. Let's look at the chart for steps 2 and 3.

We divided by 8 to get We are allowed to do this because, as Remark 29 states, we can divide by any element in the vector as long as we are consistent with our choice throughout the problem.

Notice that . This means that our vectors will just continue in a cycle and never converge. The power method only works if there is one eigenvalue whose absolute value is strictly larger than the absolute value of the other eigenvalues. Even when the power method works, convergence might be slow.

The power method found one very important eigenpair, but what should we do if we want to find all the eigenpairs? We know that for all eigenpairs. We can transform this equation into a form that will help us. When we learned about the identity matrix, we learned that Ix = x for any x. Therefore, We can use algebra steps from here.

We know that x = 0 would solve this equation, but we defined an eigenvector to be non-zero, so if there is an eigenvector solution to the equation then there must be more than one solution to the equation. We learned in Chapter 6 that the system has a unique solution if Therefore, we know that if there is a non-zero solution to then The equation even has a name. It is called the characteristic equation. We can solve the characteristic equation to find all the eigenvalues of certain matrices. There will be as many eigenvalues as there are rows in the matrix (or columns since the matrix must be square), but some of the eigenvalues might be identical to each other.

Let's find both of the eigenvalues of the matrix

Therefore, or We now know our eigenvalues. Remember that all eigenvalues are paired with an eigenvector. Therefore, we can substitute our eigenvalues, one at a time, into the formula and solve to find a corresponding eigenvector.

Let's find an eigenvector corresponding to

Notice that this system is underdetermined. Therefore, there are an infinite number of solutions. So, any vector that solves the equation x₁ - 2x₂ = 0 is an eigenvector corresponding to when To have a consistent method for finding an eigenvector, let's choose the solution in which x₂ = 1. We can use back-substitution to find that x₁ - 2(1) = 0 which implies that x₁ = 2. This tells us that is an eigenvector corresponding to when This is the same solution that we found when we used the power method to find the dominant eigenpair.

Let's find an eigenvector corresponding to

Notice that this system is underdetermined. This will always be true when we are finding an eigenvector using this method. So, any vector that solves the equation x₁ + 3x₂ = 0 is an eigenvector corresponding to the eigenvalue when Again, let's choose the eigenvector in which the last element of x is 1. Therefore, x₂ = 1 and x₁ + 3(1) = 0, so x₁ = -3. This tells us that is an eigenvector corresponding to when Using the characteristic equation and Gaussian elimination, we are able to find all the eigenvalues to the matrix and corresponding eigenvectors.

Let's find the eigenpairs for the matrix for which the power method fails.

Therefore, or . The power method does not work because |4| = |-4|. In other words, there is not a unique dominant eigenvalue.

Let's find an eigenvector corresponding to

Since the system is underdetermined, we have an infinite number of solutions. Let's choose the solution in which x₂ = 1. We can use back-substitution to find that x₁ - 3(1 ) = 0 which implies that x₁ = 3. This tells us that is an eigenvector corresponding to when

Let's find an eigenvector corresponding to

Again, let's choose the eigenvector in which the last element of x is 1. Therefore, x₂ = 1 and x₁ + 1(1) = 0, so x₁ = -1. This tells us that is an eigenvector corresponding to when Using the characteristic equation and Gaussian elimination, we are able to find both of the eigenvalues to the matrix and corresponding eigenvectors.

We can find eigenpairs for larger systems using this method, but the characteristic equation gets impossible to solve directly when the system gets too large. We could use approximations that get close to solving the characteristic equation, but there are better ways to find eigenpairs that you will study in the future. However, these two methods give you an idea of how to find eigenpairs.

Another matrix for which the power method will not work is the matrix because the eigenvalues are both the real number 5. The method that we showed you earlier will yield the eigenvector to correspond to the eigenvalue . Other methods will reveal, and you can check, that is also an eigenvector of A corresponding to . Notice that these two eigenvectors are not multiples of one another. If the same eigenvalue is repeated p times for a particular matrix, then there can be as many as p different eigenvectors that are not multiples of each other that correspond to that eigenvalue.

We said that eigenvalues are often complex numbers. However, if the matrix A is symmetric, then the eigenvalues will always be real numbers. As you can see from the problems that we worked, eigenvalues can also be real when the matrix is not symmetric, but keep in mind that they are not guaranteed to be real.

Did you know that the determinant of a matrix is related to the eigenvalues of the matrix? The product of the eigenvalues of a square matrix is equal to the determinant of that matrix. Let's look at the two matrices that we have been working with. For

You can use this as a check to see that you have the correct eigenvalues and determinant for the matrix A.

Now that we know how to find eigenpairs, we might want to know what uses they have. The interesting uses come from larger systems, so we will just discuss them rather than solve them. Have you ever seen the video of the collapse of the Tacoma Narrows Bridge? The Tacoma Bridge was built in 1940. From the beginning, the bridge would form small waves like the surface of a body of water. This accidental behavior of the bridge brought many people who wanted to drive over this moving bridge. Most people thought that the bridge was safe despite the movement. However, about four months later, the oscillations (waves) became bigger. At one point, one edge of the road was 28 feet higher than the other edge. Finally, this bridge crashed into the water below. One explanation for the crash is that the oscillations of the bridge were caused by the frequency of the wind being too close to the natural frequency of the bridge. The natural frequency of the bridge is the eigenvalue of smallest magnitude of a system that models the bridge. This is why eigenvalues are very important to engineers when they analyze structures. (Differential Equations and Their Applications, 1983, pp. 171-173).

Remark 32 The eigenvalue of smallest magnitude of a matrix is the same as the inverse (reciprocal) of the dominant eigenvalue of the inverse of the matrix. Since most applications of eigenvalues need the eigenvalue of smallest magnitude, the inverse matrix is often solved for its dominant eigenvalue. This is why the dominant eigenvalue is so important.

Also, a bridge in Manchester, England collapsed in 1831 because of conflicts between frequencies. However, this time, the natural frequency of the bridge was matched by the frequency caused by soldiers marching in step. Large oscillations occurred and the bridge collapsed. This is why soldiers break cadence when crossing a bridge.

Frequencies are also used in electrical systems. When you tune your radio, you are changing the resonant frequency until it matches the frequency at which your station is broadcasting. Engineers used eigenvalues when they designed your radio.

Frequencies are also vital in music performance. When instruments are tuned, their frequencies are matched. It is the frequency that determines what we hear as music. Although musicians do not study eigenvalues in order to play their instruments better, the study of eigenvalues can explain why certain sounds are pleasant to the ear while others sound "flat" or "sharp." When two people sing in harmony, the frequency of one voice is a constant multiple of the other. That is what makes the sounds pleasant. Eigenvalues can be used to explain many aspects of music from the initial design of the instrument to tuning and harmony during a performance. Even the concert halls are analyzed so that every seat in the theater receives a high quality sound.

Car designers analyze eigenvalues in order to damp out the noise so that the occupants have a quiet ride. Eigenvalue analysis is also used in the design of car stereo systems so that the sounds are directed correctly for the listening pleasure of the passengers and driver. When you see a car that vibrates because of the loud booming music, think of eigenvalues. Eigenvalue analysis can indicate what needs to be changed to reduce the vibration of the car due to the music.

Eigenvalues are not only used to explain natural occurrences, but also to discover new and better designs for the future. Some of the results are quite surprising. If you were asked to build the strongest column that you could to support the weight of a roof using only a specified amount of material, what shape would that column take? Most of us would build a cylinder like most other columns that we have seen. However, Steve Cox of Rice University and Michael Overton of New York University proved, based on the work of J. Keller and I. Tadjbakhsh, that the column would be stronger if it was largest at the top, middle, and bottom. At the points of the way from either end, the column could be smaller because the column would not naturally buckle there anyway. A cross-section of this column would look like this:

Does that surprise you? This new design was discovered through the study of the eigenvalues of the system involving the column and the weight from above. Note that this column would not be the strongest design if any significant pressure came from the side, but when a column supports a roof, the vast majority of the pressure comes directly from above.

Eigenvalues can also be used to test for cracks or deformities in a solid. Can you imagine if every inch of every beam used in construction had to be tested? The problem is not as time consuming when eigenvalues are used. When a beam is struck, its natural frequencies (eigenvalues) can be heard. If the beam "rings," then it is not flawed. A dull sound will result from a flawed beam because the flaw causes the eigenvalues to change. Sensitive machines can be used to "see" and "hear" eigenvalues more precisely.

Oil companies frequently use eigenvalue analysis to explore land for oil. Oil, dirt, and other substances all give rise to linear systems which have different eigenvalues, so eigenvalue analysis can give a good indication of where oil reserves are located. Oil companies place probes around a site to pick up the waves that result from a huge truck used to vibrate the ground. The waves are changed as they pass through the different substances in the ground. The analysis of these waves directs the oil companies to possible drilling sites.

There are many more uses for eigenvalues, but we only wanted to give you a sampling of their uses. When you study science or engineering in college, you will become quite familiar with eigenvalues and their uses. There are also numerical difficulties that can arise when data from real-world problems are used. Some of these difficulties are discussed in Chapter 10.