12.8 Solutions to Markov Chains Problems

Many of these problems require tedious multiplications. You might want to consider letting the students use calculators on this and the following chapters.

1. 
   

2. 1.  
      
      
      
      

   2. Information that flows into C never leaves. Since matter has a way to get to C from A and B, and because matter can never leave C once it flows into C, eventually, C should absorb all the matter in the system.
   
   

3. 1. Yes. Each element of the transition matrix is a probability. The elements of each row sum to 1. The matrix has a row and a column for each state.
      

   2. No.   .6 + .1 + .2 = .9 ≠ 1
      

   3. No.   .25 + .15 + .3 +.4 = 1.1 ≠ 1
      

4. 1. No. The probability of leaving B (including the chance of immediately returning to B) is less than 1.
      

   2. Yes.
      

5.  
   The students might arrange the rows and, therefore the columns, in a different order.
   

6. 1. 0.3
      

   2. 0.28
      

   3. 0.336
      

   4.  
      

   5.  
      

   6.  
      

   7.  
   
   

7. 1.  

      The students might arrange the rows and, therefore the columns, in a different order.
      

   2. 0.4
      

   3. 0.05
      

   4. 0.18
      

   5.  
      

   6.  
      

   7. 0.18
      

   8.  
   9. If we are looking far enough into the future (a few weeks or longer), it doesn't matter what kind of assignment we have today. We have a 49% chance of having a full assignment, a 33% chance of having a partial assignment and an 18% chance of not having an assignment.