> [!NOTE] Definition (Discrete-time, ST227) > Let $\{ X_{n} \}_{n\geq 0}$ be a discrete-time [[Stochastic Process|stochastic process]] with a countable state space $S$. We say that $\{ X_{n} \}_{n\geq 0}$ is a discrete-time Markov chain iff it satisfies the Markov property, that is $\mathbb{P}(X_{n+1}=x_{n+1} \mid X_{n} = x_{n}, X_{n-1} = x_{n-1}, \dots, X_{0}= x_{0}) = \mathbb{P}(X_{n+1} \mid X_{n}=x_{n}) ,$for any $n\in \mathbb{N}$ and $x_{0},x_{1}, \dots, x_{n+1} \in S$. **Terminology**: - We say that the Markov chain $\{ X_{n} \}_{n \geq 0}$ is **time-homogeneous** if $\mathbb{P}(X_{n+1}=a \mid X_{n} =b) = \mathbb{P}(X_{1} = a \mid X_{0} = b).$ In this case, the **one-step transition matrix** is given by $P_{ij}=\mathbb{P}(X_{1}=j \mid X_{0}=i)$, for all $i, j\in S$. (*Note a given matrix is a transition matrix iff: it is square: its entries are non-negative; and the sum of each row is $1$.*) -