Skip to content

Answer

1. The probability of eventually leaving course 6 is 1 as states 15 and 9 are absorbing states.

2. Here we have to calculate the probability of absortion into state 15. Let \(a_{i}\) denote the probability of absorption into state 15 from state \(i\). Then, \(a_{15} = 1\) and \(a_{9} = 0\). Using equations from section,

\[ \begin{aligned} a_{6-1} &= \frac{1}{2}a_{6-1} + \frac{1}{8} a_{6-2} + \frac{1}{8} a_{6-3} + \frac{1}{8}a_{9} + \frac{1}{8}a_{15}\newline a_{6-2} &= \frac{1}{2}a_{15} + \frac{3}{8}a_{6-1} + \frac{1}{8}a_{6-3}\newline a_{6-3} &= \frac{1}{4}a_{9} + \frac{3}{8}a_{6-1} + \frac{3}{8}a_{6-2} \end{aligned} \]

Solving the 3 equations, 3 variable system, \(a_{6-1} = 105/184, a_{6-2} = 143/184\) and \(a_{6-3} = 93/184\).

3. Let \(\mu_{i}\) denote the expected number of steps to get absorbed starting from state \(i\). Then, \(\mu_{15} = \mu_{9} = 0\). Using equations from section,

\[ \begin{aligned} \mu_{6-1} &= 1 + \frac{1}{2}\mu_{6-1} + \frac{1}{8} \mu_{6-2} + \frac{1}{8} \mu_{6-3} + \frac{1}{8}\mu_{9} + \frac{1}{8}\mu_{15}\newline \mu_{6-2} &= 1 + \frac{1}{2}\mu_{15} + \frac{3}{8}\mu_{6-1} + \frac{1}{8}\mu_{6-3}\newline \mu_{6-3} &= 1 + \frac{1}{4}\mu_{9} + \frac{3}{8}\mu_{6-1} + \frac{3}{8}\mu_{6-2} \end{aligned} \]

Solving, \(\mu_{6-1} = 81/23, \mu_{6-2} = 63/23\) and \(\mu_{6-3} = 77/23\).

4. This question can be done in a manner similar to the equations described above but with a small adjustment. Note that, we can either have 0, 1, or 2 ice creams. Consider \(v_{i}(j)\) as the probability of making \(j\) additional ice creams from 6-2 to 6-1 or 6-3 to 6-1 transitions, given the current state is \(i\). Note \(v_{15}(0) = v_{9}(0) = 1\). Then,

\[ \begin{aligned} v_{6-1}(0) &= \frac{1}{2}v_{6-1}(0) + \frac{1}{8} v_{6-2}(0) + \frac{1}{8} v_{6-3}(0) + \frac{1}{8}v_{9}(0) + \frac{1}{8}v_{15}(0)\newline v_{6-2}(0) &= \frac{1}{2} v_{15}(0) + \frac{3}{8}(0) + \frac{1}{8}v_{6-3}(0)\newline v_{6-3}(0) &= \frac{1}{4}v_{9}(0) + \frac{3}{8}(0) + \frac{3}{8}v_{6-2}(0) \end{aligned} \]

Some of the transitions have been directly replaced with 0 as we are considering 0 ice creams and thus those transitions are not possible (6-2 to 6-1 for instance). Solving, \(v_{6-1}(0) = 46/61, v_{6-2}(0) = 34/61\) and \(v_{6-3}(0) = 28/61\).

The same way, we can construct equations for 1 additional steps where \(v_{15}(1) = v_{9} = 0\).

\[ \begin{aligned} v_{6-1}(1) &= \frac{1}{2}v_{6-1}(1) + \frac{1}{8} v_{6-2}(1) + \frac{1}{8} v_{6-3}(1) + \frac{1}{8}v_{9}(1) + \frac{1}{8}v_{15}(1)\newline v_{6-2}(1) &= \frac{1}{2} v_{15}(1) + \frac{3}{8}v_{6-1}(0) + \frac{1}{8}v_{6-3}(1)\newline v_{6-3}(1) &= \frac{1}{4}v_{9}(1) + \frac{3}{8}v_{6-1}(0) + \frac{3}{8}v_{6-2}(1) \end{aligned} \]

In the second equation, after going from 6-2 to 6-1, we can only get 0 more ice creams. Hence, some of the values have been replaced with the \(v_{i}(0)\) calculated above. Solving, \(v_{6-1}(1) = 690/3721, v_{6-2}(1) = 1242/3721\) and \(v_{6-3}(1) = 1518/3721\).

Note that since the total ice creams are 0, 1, or 2, we have \(v_{6-1}(0) + v_{6-1}(1) + v_{6-1}(2) = 1\).

\[ \begin{aligned} E\[\text{ice creams}\] = 0 \times v_{6-1}(0) + 1 \times v_{6-1}(1) + 2 \times v_{6-1}(2) = 1140/3721 \end{aligned} \]

5. We need to recalculate the the transition probabilities since we are conditioning on the event \(A\) that we land up in state 15.

\[ \begin{aligned} P_{ij \vert A} &= P(X_{n+1}=j \vert X_{i}=i,A)\newline &= \frac{P(X_{n+1}=j, X_{n}=i, A)}{P(X_{n}=i, A)}\newline &= \frac{P(A \vert X_{n+1}=j, X_{n}=i) P(X_{n+1}=j \vert X_{n}=i) P(X_{n}=i)}{P(A \vert X_{n}=i) P(X_{n}=i)}\newline &= \frac{P(A \vert X_{n+1}=j) P(X_{n+1}=j \vert X_{n}=i)}{P(A \vert X_{n}=i)}\newline &= \frac{a_{j}}{a_{i}} P_{ij} \end{aligned} \]

where \(a_{i}\) is the probability of absorption into state 15 starting from state \(i\). Since markov process is only dependent on the last state, absoprtion probabilities are not dependent on \(n\).

We can write equations similar to section for calculating the expected number of steps with the adjusted transition probabilities

\[ \begin{aligned} \mu_{6-1} &= 1 + \frac{a_{6-1}}{a_{6-1}}\frac{1}{2} \mu_{6-1} + \frac{a_{6-2}}{a_{6-1}}\frac{1}{8} \mu_{6-2} + \frac{a_{6-3}}{a_{6-1}}\frac{1}{8} \mu_{6-3} + \frac{a_{15}}{a_{6-1}}\frac{1}{8} \mu_{15}+ \frac{a_{9}}{a_{6-1}}\frac{1}{8} \mu_{9}\newline \mu_{6-2} &= 1 + \frac{a_{6-1}}{a_{6-2}}\frac{3}{8} \mu_{6-1} + \frac{a_{6-3}}{a_{6-2}}\frac{1}{8} \mu_{6-3} + \frac{a_{15}}{a_{6-2}}\frac{1}{2} \mu_{15}\newline \mu_{6-3} &= 1 + \frac{a_{6-1}}{a_{6-3}}\frac{3}{8} \mu_{6-1} + \frac{a_{6-2}}{a_{6-3}}\frac{3}{8} \mu_{6-2} + \frac{a_{9}}{a_{6-3}}\frac{1}{4} \mu_{9}\newline \end{aligned} \]

where \(\mu_{15} = \mu_{9} = 0, a_{15} = 1\), and \(a_{9} = 0\). The absorption probabilities can be taken from part 2. Solving, \(\mu_{6-1} = 1763/483\).

6. The changed probabilites become

\[ \begin{aligned} P(X_{n+1}=15 \vert X_{n}=6-1) &= P(X_{n+1}=6-2 \vert X_{n}=6-1)\newline &= P(X_{n+1}=6-3 \vert X_{n}=6-1) = 1/6\newline P(X_{n+1}=6-1 \vert X_{n}=6-2) &= 3/4\newline P(X_{n+1}=6-3 \vert X_{n}=6-2)=1/4 \end{aligned} \]

We then use equations from section to calculate the expected values

\[ \begin{aligned} \mu_{6-1} &= 1 + \frac{1}{2}\mu_{6-1} + \frac{1}{6} \mu_{6-2} + \frac{1}{6} \mu_{6-3} + \frac{1}{6}\mu_{9}\newline \mu_{6-2} &= 1 + \frac{3}{4}\mu_{6-1} + \frac{1}{4}\mu_{6-3}\newline \mu_{6-3} &= 1 + \frac{1}{4}\mu_{9} + \frac{3}{8}\mu_{6-1} + \frac{3}{8}\mu_{6-2} \end{aligned} \]

where \(\mu_{15} = 0\). Solving, \(\mu_{6-1} = 86/13, \mu_{6-2} = 98/13\) and \(\mu_{6-3} = 82/13\).

7. If we look carefully at the new probabilities, states 15 and 9 become recurrent. Far into the future, we are sure to land up in those states, and will be in either one of those. By symmetry, the two should be same. \(\pi_{15} = \pi_{9} = 1/2\).

8. We assume that 6-1 is an absorbing state, and accordingly calculate the probabilities. Note that there will not be an equation for 6-1 since we are then already in the final state.

\[ \begin{aligned} \mu_{6-2} &= 1 + \frac{1}{8} \mu_{6-3} + \frac{1}{2} \mu_{15}\newline \mu_{6-3} &= 1 + \frac{3}{8} \mu_{6-2} + \frac{1}{4} \mu_{9}\newline \mu_{9} &= 1 + \frac{7}{8} \mu_{9}\newline \mu_{15} &= 1 + \frac{7}{8} \mu_{15}\newline \end{aligned} \]

Solving, \(\mu_{6-2} = 344/61, \mu_{6-3} = 312/61\) and \(\mu_{9} = \mu_{15} = 8\). Plugging these into the following equation (which corresponds to taking one step out of 6-1),

\[ \begin{aligned} \mu_{6-1} = 1 + \frac{1}{2} \mu_{6-1} + \frac{1}{8} \mu_{15} + \frac{1}{8} \mu_{6-2} + \frac{1}{8} \mu_{6-3} + \frac{1}{8} \mu_{15} = \frac{265}{61} \end{aligned} \]