###### GS 2000/Math - Collins
Lab 8
Markov Processes and Risk

```In this lab you will create some transition matrices and use
MATLAB to study how the probability distribution changes over time.
This is all in preparation for you to do the 2nd Risk assignment.

Simple Dice Game

Roll a single die getting the numbers 1 thru 5 in order, if you
roll a 6 before you get to 5, you have to start over.

There are 6 states you can be in: have nothing, have 1, 2, 3, 4, 5.
Let's number these 0, 1, 2, 3, 4, 5.
Here's the transition matrix for this game:
0    1    2    3    4    5
0  5/6  1/6  1/6  1/6  1/6   0
1  1/6  4/6   0    0    0    0
2   0   1/6  4/6   0    0    0
3   0    0   1/6  4/6   0    0
4   0    0    0   1/6  4/6   0
5   0    0    0    0   1/6   1

Enter this in MATLAB, call it T  (don't enter the indices 0-5)
To check it type sum(T) you should get all 1s.

Let P = [1; 0; 0; 0; 0; 0] this is your starting distribution.

Now if you type T*P you get the distribution after one turn.
If you type T^2*P you get the distribution after two turns.
If you type T^3*P you get the distribution after three turns.

Questions: (Either e-mail the answers to me, or hand them in)

1. What is the probability that you will be back at 0 after 3 turns?

2. What is the probability that you will be back at 0 after 10 turns?

3. What is the probability that you have reached 5 after 10 turns?

4. What is the average score obtained after 10 turns? (Expected Value)

5. How many turns before the probability that you have reached 5
exceeds 1/2?

Cootie  (Send the answers to me)

1. List all the possible states you could be in during a game of Cootie.

2. Assume you have Body, Head, Eyes and Ears.  Write out the transition
matrix between all the states with at least Body, Head, Eyes and Ears
(there are 4 such states).

3. If you have Body, Head, Eyes and Ears, what is the expected number
of turns until you have a complete Cootie?  Hint: take the transition
matrix T from 2. and the right starting distribution P.
If P(4) is the probability of have a complete Cootie, then
the 4th component of T*P is the prob. after 1 turn,  T^2*P
after 2 turns and so on.  The expected value is the sum of the
number of turns times the probability of success at that turn.

Risk

Do the assignment on today's handout about the game of Risk, bring
the results on Monday.

E-mail: ccollins@math.utk.edu

```
ccollins@math.utk.edu