Chapter 9, Problem 9.4PTE

Suppose that 3 white and 3 black balls are distributed in two urns in such a way that each urn contains 3 balls. We say that the system is in state i if the first urn contains i white balls. $i=0,1,2,3$. At each stage 1 ball is drawn from each urn and the ball drawn from the first urn is placed in the second, and conversely with the ball from the second urn. Let $X_n$, denote the state of the system after the nth stage, and compute the transition probabilities of the Markov chain $\left\{X_n,n\ge 0\right\}$.