MMiDS 8.2: Self-Assessment Quiz

Let \( \mathbf{a} = (a_1,\ldots,a_n), \mathbf{b} = (b_1,\ldots,b_n), \mathbf{c} = (c_1,\ldots,c_n) \in \mathbb{R}^n \). Which of the following is true about the Hadamard product \( \odot \)?

a) \( \mathbf{a}^T(\mathbf{b} \odot \mathbf{c}) = (\mathbf{a}^T \mathbf{b}) \odot (\mathbf{a}^T \mathbf{c}) \)
b) \( \mathbf{a}^T(\mathbf{b} \odot \mathbf{c}) = \mathbf{1}^T(\mathbf{a} \odot \mathbf{b} \odot \mathbf{c}) \)
c) \( \mathbf{a}^T(\mathbf{b} \odot \mathbf{c}) = \mathbf{a}^T(\mathbf{b} \otimes \mathbf{c}) \)
d) \( \mathbf{a}^T(\mathbf{b} \odot \mathbf{c}) = (\mathbf{a} \odot \mathbf{b})^T \mathbf{c} \)

Let \( A \in \mathbb{R}^{n \times m} \) and \( B \in \mathbb{R}^{p \times q} \). What are the dimensions of the Kronecker product \( A \otimes B \)?

a) \( n \times m \)
b) \( p \times q \)
c) \( np \times mq \)
d) \( nq \times mp \)

What is the result of the Kronecker product \( A \otimes B \) if \( A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \) and \( B = \begin{pmatrix} 0 & 5 \\ 6 & 7 \end{pmatrix} \)?

a) \( \begin{pmatrix} 1 \otimes 0 & 2 \otimes 5 \\ 3 \otimes 6 & 4 \otimes 7 \end{pmatrix} \)
b) \( \begin{pmatrix} 0 & 5 & 0 & 10 \\ 6 & 7 & 12 & 14 \\ 0 & 15 & 0 & 20 \\ 18 & 21 & 24 & 28 \end{pmatrix} \)
c) \( \begin{pmatrix} 0 & 5 & 5 & 10 \\ 6 & 7 & 7 & 14 \\ 18 & 21 & 21 & 28 \\ 24 & 28 & 28 & 32 \end{pmatrix} \)
d) \( \begin{pmatrix} 0 & 10 & 10 & 20 \\ 12 & 14 & 14 & 28 \\ 18 & 30 & 30 & 40 \\ 24 & 28 & 28 & 32 \end{pmatrix} \)

If \( f: \mathbb{R}^d \rightarrow \mathbb{R}^m \) is a continuously differentiable function, what is its Jacobian \( J_f(x_0) \) at an interior point \( x_0 \) of its domain?

a) A scalar representing the rate of change of \( f \) at \( x_0 \).
b) A vector in \( \mathbb{R}^m \) representing the direction of steepest ascent of \( f \) at \( x_0 \).
c) An \( m \times d \) matrix of partial derivatives of the component functions of \( f \) at \( x_0 \).
d) The Hessian matrix of \( f \) at \( x_0 \).

Let \( f: \mathbb{R}^d \rightarrow \mathbb{R} \) be a twice continuously differentiable function. What is the Jacobian of its gradient \( \nabla f \)?

a) The gradient of \( f \).
b) The Hessian of \( f \).
c) The Laplacian of \( f \).
d) The directional derivative of \( f \).

What is the relationship between the Jacobian of a continuously differentiable real-valued function \( f: \mathbb{R}^d \rightarrow \mathbb{R} \) and its gradient \( \nabla f \)?

a) The Jacobian is the transpose of the gradient.
b) The Jacobian is the negative of the gradient.
c) The Jacobian is the gradient.
d) The Jacobian is the magnitude of the gradient.

In the context of the Chain Rule, if \( f: \mathbb{R}^2 \to \mathbb{R}^3 \) and \( g: \mathbb{R}^3 \to \mathbb{R} \), what is the dimension of the Jacobian matrix \( J_{g \circ f}(x) \)?

a) \( 3 \times 2 \)
b) \( 1 \times 3 \)
c) \( 2 \times 3 \)
d) \( 1 \times 2 \)

Let \( \mathbf{f} : D_1 \to \mathbb{R}^m \), where \( D_1 \subseteq \mathbb{R}^d \), and let \( \mathbf{g} : D_2 \to \mathbb{R}^p \), where \( D_2 \subseteq \mathbb{R}^m \). Assume that \( \mathbf{f} \) is continuously differentiable at \( \mathbf{x}_0 \), an interior point of \( D_1 \), and that \( \mathbf{g} \) is continuously differentiable at \( \mathbf{f}(\mathbf{x}_0) \), an interior point of \( D_2 \). Which of the following is correct according to the Chain Rule?

a) \( J_{\mathbf{g} \circ \mathbf{f}}(\mathbf{x}_0) = J_{\mathbf{f}}(\mathbf{x}_0) \, J_{\mathbf{g}}(\mathbf{f}(\mathbf{x}_0)) \)
b) \( J_{\mathbf{g} \circ \mathbf{f}}(\mathbf{x}_0) = J_{\mathbf{g}}(\mathbf{f}(\mathbf{x}_0)) \, J_{\mathbf{f}}(\mathbf{x}_0) \)
c) \( J_{\mathbf{g} \circ \mathbf{f}}(\mathbf{x}_0) = J_{\mathbf{f}}(\mathbf{g}(\mathbf{x}_0)) \, J_{\mathbf{g}}(\mathbf{x}_0) \)
d) \( J_{\mathbf{g} \circ \mathbf{f}}(\mathbf{x}_0) = J_{\mathbf{g}}(\mathbf{x}_0) \, J_{\mathbf{f}}(\mathbf{g}(\mathbf{x}_0)) \)

Given the function \( f(x) = \begin{pmatrix} x_1^2 \\ \sin(x_2) \end{pmatrix} \), what is the Jacobian \( J_f \) at the point \( (1, \frac{\pi}{2}) \)?

a) \( \begin{pmatrix} 2 & 0 \\ 0 & \cos(x_2) \end{pmatrix} \)
b) \( \begin{pmatrix} 2 & 0 \\ 0 & 1 \end{pmatrix} \)
c) \( \begin{pmatrix} 2 & 0 \\ 0 & 0 \end{pmatrix} \)
d) \( \begin{pmatrix} 1 & 0 \\ 0 & \cos(\frac{\pi}{2}) \end{pmatrix} \)

Let \( f: \mathbb{R}^2 \rightarrow \mathbb{R}^3 \) be defined by \( f(x_1, x_2) = (3x_1^2, x_2, x_1x_2)^T \). What is the Jacobian matrix \( J_f(x_1, x_2) \)?

a) \( \begin{pmatrix} 6x_1 & 0 \\ 0 & 1 \\ x_2 & x_1 \end{pmatrix} \)
b) \( \begin{pmatrix} 3x_1^2 & x_2 \\ x_2 & x_1x_2 \end{pmatrix} \)
c) \( \begin{pmatrix} 6x_1 & 1 \\ x_2 & x_1 \end{pmatrix} \)
d) \( \begin{pmatrix} 6x_1 & x_1 \\ 0 & x_2 \end{pmatrix} \)

Let \( A \in \mathbb{R}^{m \times d} \) and \( \mathbf{b} \in \mathbb{R}^{m} \). Define the vector-valued function \( \mathbf{f} : \mathbb{R}^d \to \mathbb{R}^m \) as \( \mathbf{f}(\mathbf{x}) = A \mathbf{x} + \mathbf{b} \). What is the Jacobian of \( \mathbf{f} \) at \( \mathbf{x}_0 \)?

a) \( J_{\mathbf{f}}(\mathbf{x}_0) = A^T \)
b) \( J_{\mathbf{f}}(\mathbf{x}_0) = A \mathbf{x}_0 + \mathbf{b} \)
c) \( J_{\mathbf{f}}(\mathbf{x}_0) = A \)
d) \( J_{\mathbf{f}}(\mathbf{x}_0) = \mathbf{b} \)

Consider the following PyTorch code:

                
x1 = torch.tensor(1.0, requires_grad=True)  
x2 = torch.tensor(2.0, requires_grad=True)
f = 3 * (x1 ** 2) + x2 + torch.exp(x1 * x2)

What does the option requires_grad=True do?

a) It indicates that \( x1 \) and \( x2 \) are constants.
b) It indicates that \( x1 \) and \( x2 \) are variables with respect to which a gradient will be computed later.
c) It indicates that \( x1 \) and \( x2 \) are random variables.
d) It indicates that \( x1 \) and \( x2 \) are the outputs of the function \( f \).

Which of the following PyTorch functions is used to compute gradients?

a) .requires_grad()
b) .backward()
c) .grad()
d) .autograd()