A Markov process is a discrete-time stochastic process. To constuct an MP, 3 components are necessary:
A Markov process is a set of random variables indexed by time \(\{X_t\}_{t=1, 2, 3...}\) where the distribution is given by:
\[\begin{align*} X_1 &\sim p_0\\ X_{t+1} \lvert X_{t} &\sim T X_t \end{align*}\]If the set \(\mathcal{X}\) is finite, the transition operator can be written as a \(\lvert \mathcal{X} \lvert \times \lvert \mathcal{X} \lvert\) dimensional matrix called a transition matrix or stochastic matrix.
Each column of \(T\) must sum to 1
The marginal probability of the \(k\)th variable \(X_k\) is given by