MMiDS 6.4: Self-Assessment Quiz

In the mixture of multivariate Bernoullis model, the joint distribution is given by:

• a) \(P[X = x] = \prod_{k=1}^K P[C = k, X = x]\)
• b) \(P[X = x] = \sum_{k=1}^K P[X = x|C = k] P[C = k]\)
• c) \(P[X = x] = \prod_{k=1}^K P[X = x|C = k] P[C = k]\)
• d) \(P[X = x] = \sum_{k=1}^K P[C = k, X = x]\)

The majorization-minimization principle states that:

• a) If \(U_x\) majorizes \(f\) at \(x\), then a global minimum \(x'\) of \(U_x\) satisfies \(f(x') \geq f(x)\).
• b) If \(U_x\) majorizes \(f\) at \(x\), then a global minimum \(x'\) of \(U_x\) satisfies \(f(x') \leq f(x)\).
• c) If \(U_x\) minorizes \(f\) at \(x\), then a global minimum \(x'\) of \(U_x\) satisfies \(f(x') \geq f(x)\).
• d) If \(U_x\) minorizes \(f\) at \(x\), then a global minimum \(x'\) of \(U_x\) satisfies \(f(x') \leq f(x)\).

In the EM algorithm for mixtures of multivariate Bernoullis, the E-step involves:

• a) Updating the parameters \(\pi_k\) and \(p_{k,m}\)
• b) Computing the responsibilities \(r_{k,i}^t\)
• c) Minimizing the negative log-likelihood
• d) Applying the log-sum-exp trick

In the EM algorithm for mixtures of multivariate Bernoullis, the M-step involves:

• a) Updating the parameters \(\pi_k\) and \(p_{k,m}\)
• b) Computing the responsibilities \(r_{k,i}^t\)
• c) Minimizing the negative log-likelihood
• d) Applying the log-sum-exp trick

The mixture of multivariate Bernoullis model is represented by the following graphical model:

• a)

  G = nx.DiGraph()
  G.add_node("X", shape="circle", style="filled", fillcolor="gray")
  G.add_node("C", shape="circle", style="filled", fillcolor="white")
  G.add_edge("C", "X")

• b)

  G = nx.DiGraph()
  G.add_node("X", shape="circle", style="filled", fillcolor="white")
  G.add_node("C", shape="circle", style="filled", fillcolor="gray")
  G.add_edge("C", "X")

• c)

  G = nx.DiGraph()
  G.add_node("X", shape="circle", style="filled", fillcolor="gray")
  G.add_node("C", shape="circle", style="filled", fillcolor="gray")
  G.add_edge("C", "X")

• d)

  G = nx.DiGraph()
  G.add_node("X", shape="circle", style="filled", fillcolor="white")
  G.add_node("C", shape="circle", style="filled", fillcolor="white")
  G.add_edge("C", "X")

In the context of the EM algorithm, what is the purpose of the majorization-minimization principle?

• a) To find the global minimum of the log-likelihood function directly.
• b) To iteratively improve a lower bound on the log-likelihood function.
• c) To avoid getting stuck in local minima of the log-likelihood function.
• d) To simplify the computation of the responsibilities in the E-step.

In a mixture of multivariate Bernoullis, what do the parameters \(p_{k,m}\) represent?

• a) The probability of observing class \(k\) given feature \(m\).
• b) The probability of observing feature \(m\) given class \(k\).
• c) The prior probability of class \(k\).
• d) The mixing weight of class \(k\).

In the context of clustering, what is the interpretation of the responsibilities computed in the E-step of the EM algorithm?

• a) They represent the distance of each data point to the cluster centers.
• b) They indicate the probability of each data point belonging to each cluster.
• c) They determine the optimal number of clusters.
• d) They are used to initialize the cluster centers in the M-step.

What is the main difference between the Naive Bayes model and the mixture of multivariate Bernoullis model?

• a) The Naive Bayes model assumes that the features are independent given the class, while the mixture of multivariate Bernoullis model does not.
• b) The Naive Bayes model is a generative model, while the mixture of multivariate Bernoullis model is a discriminative model.
• c) The Naive Bayes model requires knowing the true labels of the data points, while the mixture of multivariate Bernoullis model does not.
• d) The Naive Bayes model can only handle binary features, while the mixture of multivariate Bernoullis model can handle categorical features.