MMiDS 8.5: Self-Assessment Quiz

What is the role of the sigmoid function in a multilayer perceptron (MLP)?

• a) It is used as the loss function for training the MLP.
• b) It is used as the nonlinear activation function in each layer of the MLP.
• c) It is used to compute the gradient of the loss function with respect to the weights.
• d) It is used to initialize the weights of the MLP.

In an MLP, what is the purpose of the softmax function in the output layer?

• a) To introduce nonlinearity into the model.
• b) To normalize the outputs into a probability distribution.
• c) To compute the gradient of the loss function.
• d) To reduce the dimensionality of the output.

What is the derivative of the sigmoid function \( \sigma(t) = \frac{1}{1 + e^{-t}} \)?

• a) \( \sigma'(t) = \sigma(t)(1 - \sigma(t)) \)
• b) \( \sigma'(t) = e^{-t} \)
• c) \( \sigma'(t) = -\sigma(t)^2 \)
• d) \( \sigma'(t) = \frac{1}{1 + e^{-t}} \)

What is the Jacobian matrix of the elementwise sigmoid function \( \sigma(t) = (\sigma(t_1), \dots, \sigma(t_n)) \)?

• a) \( J_\sigma(t) = \mathrm{diag}(\sigma(t)) \)
• b) \( J_\sigma(t) = \sigma(t) \odot (1-\sigma(t)) \)
• c) \( J_\sigma(t) = \mathrm{diag}(\sigma(t) \odot (1 - \sigma(t))) \)
• d) \( J_\sigma(t) = \sigma(t)(1 - \sigma(t))^T \)

In the forward phase of computing the gradient of the loss function in an MLP, what is the output of the \( i \)-th hidden layer?

• a) \( z_{i+1} := g_i(z_i, w_i) = \sigma(W_i z_i) \)
• b) \( z_{i+1} := g_i(z_i, w_i) = W_i z_i \)
• c) \( z_{i+1} := g_i(z_i, w_i) = \gamma(W_i z_i) \)
• d) \( z_{i+1} := g_i(z_i, w_i) = W_i \sigma(z_i) \)

What is the output of the backward loop in computing the gradient of the loss function in an MLP?

• a) The gradient of the loss function with respect to the activations of each layer.
• b) The gradient of the loss function with respect to the weights of each layer.
• c) The updated weights of the MLP.
• d) The loss function value.

What is the purpose of the convolutional layers in a CNN?

• a) To fully connect each neuron in the current layer to all neurons in the previous layer.
• b) To apply a nonlinear activation function to the input tensor.
• c) To reduce the spatial dimensions of the input tensor.
• d) To learn local patterns and features from the input tensor.