MMiDS 7.2: Self-Assessment Quiz

Which of the following is true about the transition matrix \(P\) of a Markov chain?

• a) All entries of \(P\) are non-negative, and all columns sum to one.
• b) All entries of \(P\) are non-negative, and all rows sum to one.
• c) All entries of \(P\) are non-negative, and both rows and columns sum to one.
• d) All entries of \(P\) are non-negative, and either rows or columns sum to one, but not both.

Which of the following is true about the transition graph of a Markov chain with transition matrix \(P\)?

• a) There is a directed edge from state \(i\) to state \(j\) if and only if \(p_{i,j} > 0\).
• b) There is a directed edge from state \(i\) to state \(j\) if and only if \(p_{i,j} = 0\).
• c) There is an undirected edge between state \(i\) and state \(j\) if and only if \(p_{i,j} > 0\).
• d) There is an undirected edge between state \(i\) and state \(j\) if and only if \(p_{i,j} = 0\).

What is the Markov property?

• a) The past and future are independent.
• b) The past and future are independent given the present.
• c) The present and future are independent given the past.
• d) The past, present, and future are all independent.

Consider a Markov chain \((X_t)_{t \ge 0}\) on state space \(S\). Which of the following equations is a direct consequence of the Markov property?

• a) \(\mathbb{P}[X_{t+1} = x_{t+1} | X_t = x_t] = \mathbb{P}[X_{t+1} = x_{t+1}]\)
• b) \(\mathbb{P}[X_{t+1} = x_{t+1} | X_t = x_t, X_{t-1} = x_{t-1}] = \mathbb{P}[X_{t+1} = x_{t+1} | X_t = x_t]\)
• c) \(\mathbb{P}[X_{t+1} = x_{t+1} | X_t = x_t] = \mathbb{P}[X_{t+1} = x_{t+1} | X_0 = x_0]\)
• d) \(\mathbb{P}[X_{t+1} = x_{t+1} | X_t = x_t, X_{t-1} = x_{t-1}] = \mathbb{P}[X_{t+1} = x_{t+1}]\)

Consider a Markov chain \((X_t)_{t\geq0}\) with transition matrix \(P\) and initial distribution \(\mu\). Which of the following is true about the distribution of a sample path \((X_0, X_1, \ldots, X_T)\)?

• a) \(P[X_0 = x_0, X_1 = x_1, \ldots, X_T = x_T] = \mu_{x_0} \prod_{t=1}^T p_{x_{t-1},x_t}\)
• b) \(P[X_0 = x_0, X_1 = x_1, \ldots, X_T = x_T] = \mu_{x_0} \sum_{t=1}^T p_{x_{t-1},x_t}\)
• c) \(P[X_0 = x_0, X_1 = x_1, \ldots, X_T = x_T] = \prod_{t=0}^T \mu_{x_t}\)
• d) \(P[X_0 = x_0, X_1 = x_1, \ldots, X_T = x_T] = \sum_{t=0}^T \mu_{x_t}\)

In the weather model example, if today is dry, what is the probability that tomorrow will also be dry?

• a) 1/4
• b) 1/2
• c) 3/4
• d) 1

What is the initial distribution vector \(\mu\) for the weather model if the weather starts as dry?

• a) \(\mu = (0.5, 0.5)\)
• b) \(\mu = (1, 0)\)
• c) \(\mu = (0, 1)\)
• d) \(\mu = (0.25, 0.75)\)

Using the transition matrix \(P\) for the weather model, what is the probability that it is dry after two days if it starts as dry?

• a) \(\frac{1}{4}\)
• b) \(\frac{1}{2}\)
• c) \(\frac{5}{8}\)
• d) \(\frac{3}{4}\)

In the random walk on the Petersen graph example, if the current state is vertex 9, what is the probability of transitioning to vertex 4 in the next step?

• a) 0
• b) 1/10
• c) 1/3
• d) 1

Which of the following is true about a doubly stochastic matrix?

• a) All entries are non-negative, and all rows sum to one.
• b) All entries are non-negative, and all columns sum to one.
• c) All entries are non-negative, and both rows and columns sum to one.
• d) All entries are non-negative, and either rows or columns sum to one, but not both.