Definition 3.1.1. Communicating states Let $\left(X_n\right)_{n \geq 0}$ be a Markov chain with the state space $S$ and the transition matrix $P$. Let $i, j \in S$. We say that state $j$ is accessible from state $i$ and write $i \rightarrow j$ if there exists $n \in \mathbb{N} \cup\{0\}$ such that $P_{i j}^n>0$. If $i \rightarrow j$ and $j \rightarrow i$ then we say that the states $i$ and $j$ communicate and write $i \leftrightarrow j$. Therefore, if state $j$ is accessible from state $i, i, j \in S, i \neq j$, then there is a positive probability that the chain starting from $i$ will reach state $j$ at some point in future. Observe that $P^0=I$, where $I$ is the identity matrix. Hence, for any $i \in S, P_{i i}^0=1>0$ which by the above definition implies that $i \rightarrow i$ regardless of whether $P_{i i}^n>0$ for any $n \in \mathbb{N}$. Lemma 3.1.1. Equivalence relation Let $\left(X_n\right)_{n \geq 0}$ be a Markov chain with the state space $S$ and the transition matrix $P$. The relation " $i$ and $j$ communicate" $(i \leftrightarrow j)$ is an equivalence relation on $S$. Proof. See ST227 Tutorial sheet 1. Recall that a property of an equivalence relation on a set is that it partitions the set into equivalence classes. Hence, the equivalence relation "i and $j$ communicate" $(i \leftrightarrow j)$ on the state space $S$ partitions $S$ into equivalence classes. We define communicating classes to be the equivalence classes formed by the relation " $i$ and $j$ communicate" $(i \leftrightarrow j)$ : Definition 3.1.2. Communicating class Let $\left(X_n\right)_{n \geq 0}$ be a Markov chain with the state space $S$ and the transition matrix $P$. A communicating class $C \subseteq S$ is a maximal collection of communicating states such that for any $i, j \in C, i \leftrightarrow j$, and for any $k \notin C$ there does not exist a state in $C$ which communicates with $k$.