Steady State Probabilities

Do $r_{ij}(n)$ converge to some ${\pi }_j$ (independent of i) ? where ${\pi }_j$ denotes the steady state probability of occupancy of state $j$, or $P(X_n=j)$ for large $n$.

Yes if,

• recurrent states are all in a single class
• single recurrent class is not periodic (otherwise oscillations are possible)

Assuming yes, $$\begin{array}{rl}{r}_{ij}(n)& =\sum _k{r}_{ik}(n-1)p_{kj}\\ \underset{n\to \mathrm{\infty }}{lim}{r}_{ij}(n)& =\underset{n\to \mathrm{\infty }}{lim}\sum _k{r}_{ik}(n-1)p_{kj}\\ {\pi }_j& =\sum _k{\pi }_k{p}_{kj}\text{balance equations}\\ \text{and,}\sum _i{\pi }_i& =1\\ \text{frequency of transitions }k\to j& ={\pi }_k{p}_{kj}\text{in one step}\\ \text{frequency of transitions into }j& =\sum _k{\pi }_k{p}_{kj}\text{influx from all connected states}\end{array}$$

The $p{i}_j$ sum up to 1 and form a probability distribution called the stationary distribution of the chain (because if the initial distribution $P(X_0=j)={\pi }_j$, the occupancy distribution of the states is constant for all steps and can be verified using total probability theorem on any of the nodes).

In the steady state,

• ${\pi }_j=0$ for transient states
• ${\pi }_j>0$ for recurrent states (note that any state that is absorbing is actually recurrent since its only connected to itself and hence accessible to itself from itself)