MMiDS 7.3: Self-Assessment Quiz

Which of the following is the correct condition for a probability distribution \(\pi = (\pi_i)_{i=1}^n\) to be a stationary distribution of a Markov chain with transition matrix \(P = (p_{i,j})_{i,j=1}^n\)?

• a) \(\sum_{j=1}^n \pi_i p_{i,j} = \pi_j\) for all \(i \in [n]\)
• b) \(\sum_{i=1}^n \pi_i p_{i,j} = \pi_j\) for all \(j \in [n]\)
• c) \(\sum_{j=1}^n \pi_i p_{i,j} = \pi_i\) for all \(i \in [n]\)
• d) \(\sum_{i=1}^n \pi_i p_{i,j} = \pi_i\) for all \(j \in [n]\)

Which of the following is the matrix form of the condition for a probability distribution \(\pi\) to be a stationary distribution of a Markov chain with transition matrix \(P\)?

• a) \(\pi P = \pi\)
• b) \(P \pi = \pi\)
• c) \(\pi P^T = \pi^T\)
• d) \(P^T \pi^T = \pi^T\)

Consider a Markov chain with the following transition matrix:

\[ P = \begin{pmatrix} 1/2 & 1/2 \\ 1/3 & 2/3 \end{pmatrix} \]

Which of the following is a stationary distribution for this Markov chain?

• a) \((1/2, 1/2)\)
• b) \((1/3, 2/3)\)
• c) \((2/5, 3/5)\)
• d) \((1, 0)\)

A Markov chain is irreducible if:

• a) Every state communicates with every other state.
• b) There exists a state that communicates with every other state.
• c) The transition graph of the chain is strongly connected.
• d) Both a and c.

Consider the following transition graph of a Markov chain:

G = nx.DiGraph()
G.add_edges_from([(0, 1), (1, 2), (2, 0), (1, 1)])

Is this Markov chain irreducible?

• a) Yes
• b) No
• c) Not enough information
• d) The concept of irreducibility does not apply to this chain.

Consider the following transition graph of a Markov chain:

G = nx.DiGraph()
G.add_edges_from([(1, 2), (2, 1), (2, 3), (3, 3)])

Is this Markov chain irreducible?

• a) Yes
• b) No

According to the Existence of Stationary Distribution Theorem, an irreducible Markov chain on a finite state space:

• a) Has a unique stationary distribution with all entries being strictly positive.
• b) May have multiple stationary distributions, but all of them have strictly positive entries.
• c) Has a unique stationary distribution, but some entries may be zero.
• d) May not have any stationary distribution.

In an irreducible Markov chain, the left and right eigenvectors corresponding to eigenvalue 1 are:

• a) The same up to scaling.
• b) Always different.
• c) Transposes of each other.
• d) Not necessarily related to each other.

For an irreducible Markov chain, which of the following statements is true?

• a) All eigenvalues of the transition matrix have absolute value strictly less than 1.
• b) All eigenvalues of the transition matrix have absolute value less than or equal to 1, and eigenvalue 1 has geometric multiplicity 1.
• c) All eigenvalues of the transition matrix have absolute value less than or equal to 1, and eigenvalue -1 has geometric multiplicity 1.
• d) The transition matrix may have eigenvalues with absolute value greater than 1.

Given the transition matrix \(P\) of a Markov chain, which method can be used to numerically find the stationary distribution?

• a) Solving the linear system \(\pi (P - I) = 0\).
• b) Computing the eigenvector corresponding to the eigenvalue 1 of \(P\).
• c) Both a and b.
• d) None of the above.