Markov Processes and Transition Matrices

In this lab you will create some transition matrices and use MATLAB to study how the probability distribution changes over time.ccollins@math.utk.eduSimple Dice GameHere's a simple game one person can play: Roll a single die getting the numbers 1 thru 5 in order, if you roll a 6 before you get to 5, you have to start over. Here's how to set up the transition matrix: There are 6 states you can be in: have nothing, have 1, 2, 3, 4, 5. Let's number these 0, 1, 2, 3, 4, 5. Then until you get to 5, you have a 1/6 chance of moving to the next state and a 1/6 chance of going back to 0. Thus the transition matrix looks like: 0 1 2 3 4 5 0 5/6 1/6 1/6 1/6 1/6 0 1 1/6 4/6 0 0 0 0 2 0 1/6 4/6 0 0 0 3 0 0 1/6 4/6 0 0 4 0 0 0 1/6 4/6 0 5 0 0 0 0 1/6 1 Enter this in MATLAB, call itT(don't enter the indices 0-5) To check it typesum(T)you should get all 1s. Let P = [1; 0; 0; 0; 0; 0] this is your starting distribution. You can also create it by P = zeros(6,1); P(1) = 1; Now if you type T*P you get the distribution after one turn. If you type T^2*P you get the distribution after two turns. If you type T^3*P you get the distribution after three turns.Questions:(Either e-mail the answers to me, or hand them in) 1. What is the probability that you will be back at 0 after 3 turns? 2. What is the probability that you will be back at 0 after 10 turns? 3. What is the probability that you have reached 5 after 10 turns? 4. What is the average score obtained after 10 turns? (Expected Value) 5. How many turns before the probability that you have reached 5 exceeds 1/2?Two Player Dice BattleThis is the game we played in class. There are 2 players, A and B. For each round, each rolls a dice, if A rolls a number greater thanor equal toB then A wins. Otherwise B wins. The first to get 3 wins is the winner. Building the transition matrix: List all the possible states the game could be in. For example, you start with A having 0 wins and B having 0 wins (write as (0,0)), at some time A could have 2 wins and B have 1 win (write as (2,1)). Number the states from 1 to N, where N is the number of states. Build the matrix T (it will be NxN). Use the probabilities we determined in class and your numbering of the states to fill in the values. Remember that a winning state (either player has 3 wins) is an absorbing state. I'd suggest you put it in a m-file "dice.m" and start it like: N = T = zeros(N); % starts T out as an NxN matrix of zeros % 1 = (0,0) % list what all the states are % 2 = .. % % N = .. T( , ) = % fill in all the non-zero values You could also enter T as T = [0 0 0 ... 0 0whatevermore stuff]; Each column (except for winning states) should have 2 entries: one if A wins and one if B wins. To check your result, type dice and then type sum(T) to see if it is all 1s.Questions:(Either e-mail the answers to me, or hand them in) 1. What is the probability of being at (2,0) after 2 rounds? (0,2)? How about (3,0) after 3 rounds? 2. What is the probability of being down (0,2) and then winning (3,2)? 3. How many rounds do you have to play before you are guaranteed to have a winner? 4. What is the probability that A wins? 5. What is the expected number of rounds before there is a winner? E-mail: ccollins@math.utk.edu