MMiDS 3.3: Self-Assessment Quiz

Which of the following is the correct definition of a global minimizer \( x^* \) of a function \( f: \mathbb{R}^d \to \mathbb{R} \)?

• a) \( f(x) \geq f(x^*) \) for all \( x \) in some open ball around \( x^* \).
• b) \( f(x) \geq f(x^*) \) for all \( x \in \mathbb{R}^d \).
• c) \( \nabla f(x^*) = 0 \).
• d) \( v^T H_f(x^*) v > 0 \) for all \( v \in \mathbb{R}^d \).

Which of the following statements about descent directions is true?

• a) A descent direction for a function \( f \) at a point \( x_0 \) is any direction in which \( f \) increases.
• b) A descent direction for a function \( f \) at a point \( x_0 \) is any direction in which \( f \) does not change.
• c) A descent direction for a function \( f \) at a point \( x_0 \) is any direction in which \( f \) decreases.
• d) A descent direction for a function \( f \) at a point \( x_0 \) is always the direction of steepest descent.

Which of the following is the correct definition of a local minimizer \( x^* \) of a function \( f: \mathbb{R}^d \to \mathbb{R} \)?

• a) \( f(x^*) \leq f(x) \) for all \( x \in \mathbb{R}^d \).
• b) \( f(x^*) < f(x) \) for all \( x \in \mathbb{R}^d \).
• c) There exists \( \delta > 0 \) such that \( f(x^*) \leq f(x) \) for all \( x \in B_\delta(x^*) \setminus \{x^*\} \).
• d) There exists \( \delta > 0 \) such that \( f(x^*) < f(x) \) for all \( x \in B_\delta(x^*) \setminus \{x^*\} \).

Which of the following statements about stationary points is true?

• a) All stationary points are local minimizers.
• b) All local minimizers are stationary points.
• c) A stationary point is a point where the gradient of the function is nonzero.
• d) A stationary point is always a saddle point.

Which of the following is a necessary condition for a point \( x^* \) to be a local minimizer of a continuously differentiable function \( f: \mathbb{R}^d \to \mathbb{R} \)?

• a) \( \nabla f(x^*) \neq 0 \).
• b) \( \nabla f(x^*) = 0 \).
• c) \( H_f(x^*) \) is positive definite.
• d) \( H_f(x^*) \) is negative definite.

Consider a continuously differentiable function \( f: \mathbb{R}^d \to \mathbb{R} \). If \( v \in \mathbb{R}^d \) is a descent direction for \( f \) at \( x_0 \), then which of the following is true?

• a) \( \nabla f(x_0)^T v > 0 \).
• b) \( \nabla f(x_0)^T v = 0 \).
• c) \( \nabla f(x_0)^T v < 0 \).
• d) None of the above.

Let \( f: \mathbb{R}^d \to \mathbb{R} \) be continuously differentiable at \( x_0 \). The directional derivative of \( f \) at \( x_0 \) in the direction \( v \in \mathbb{R}^d \) is NOT given by:

• a) \( \frac{\partial f(x_0)}{\partial v} = \nabla f(x_0)^T v \).
• b) \( \frac{\partial f(x_0)}{\partial v} = v^T \nabla f(x_0) \).
• c) \( \frac{\partial f(x_0)}{\partial v} = v^T H_f(x_0) v \).
• d) \( \frac{\partial f(x_0)}{\partial v} = \lim_{h \to 0} \frac{f(x_0 + hv) - f(x_0)}{h} \).

Let \( f: \mathbb{R}^d \to \mathbb{R} \) be twice continuously differentiable at \( x_0 \). The second directional derivative of \( f \) at \( x_0 \) in the direction \( v \in \mathbb{R}^d \) is given by:

• a) \( \frac{\partial^2 f(x_0)}{\partial v^2} = \lim_{h \to 0} \frac{\nabla f(x_0 + hv) - \nabla f(x_0)}{h} \).
• b) \( \frac{\partial^2 f(x_0)}{\partial v^2} = \lim_{h \to 0} \frac{1}{h} [\frac{\partial f(x_0 + hv)}{\partial v} - \frac{\partial f(x_0)}{\partial v}] \).
• c) \( \frac{\partial^2 f(x_0)}{\partial v^2} = v^T H_f(x_0) v \).
• d) Both b and c.

Let \( f: \mathbb{R}^d \to \mathbb{R} \) be twice continuously differentiable. If \( \nabla f(x_0) = 0 \) and \( H_f(x_0) \) is positive definite, then \( x_0 \) is:

• a) A global minimizer of \( f \).
• b) A local minimizer of \( f \), but not necessarily a strict local minimizer.
• c) A strict local minimizer of \( f \).
• d) A saddle point of \( f \).

What is the Lagrangian function used for in constrained optimization?

• a) It transforms a constrained optimization problem into an unconstrained one.
• b) It provides a way to find the feasible set of the constrained problem.
• c) It directly gives the optimal solution to the constrained problem.
• d) It is used to calculate the gradient of the objective function.

In the method of Lagrange multipliers, what is the geometric interpretation of the condition \( \nabla f(x^*) + \sum_{i=1}^l \lambda_i^* \nabla h_i(x^*) = 0 \)?

• a) The gradient of the objective function is parallel to the gradient of each constraint function.
• b) The gradient of the objective function is orthogonal to the subspace of first-order feasible directions at \( x^* \).
• c) The gradient of the objective function is zero at \( x^* \).
• d) The Lagrange multipliers are all equal to zero.

Consider the optimization problem \( \min_x f(x) \) subject to \( h(x) = 0 \), where \( f: \mathbb{R}^d \to \mathbb{R} \) and \( h: \mathbb{R}^d \to \mathbb{R}^\ell \) are continuously differentiable. Let \( x^* \) be a local minimizer and assume that the vectors \( \nabla h_i(x^*), i \in [\ell] \), are linearly independent. According to the Lagrange Multipliers theorem, which of the following must be true?

• a) \( \nabla f(x^*) = 0 \).
• b) \( \nabla f(x^*) + \sum_{i=1}^\ell \lambda_i^* \nabla h_i(x^*) = 0 \) for some \( \lambda^* \in \mathbb{R}^\ell \).
• c) \( h(x^*) = 0 \).
• d) Both b and c.

Consider the optimization problem \( \min_x f(x) \) subject to \( h(x) = 0 \), where \( f: \mathbb{R}^d \to \mathbb{R} \) and \( h: \mathbb{R}^d \to \mathbb{R}^\ell \) are continuously differentiable. Let \( x^* \) be a local minimizer satisfying the first-order necessary conditions with Lagrange multipliers \( \lambda^* \). The subspace of first-order feasible directions at \( x^* \) is defined as:

• a) \( F_h(x^*) = \{v \in \mathbb{R}^d: \nabla h_i(x^*)^T v = 0, \forall i \in [\ell]\} \).
• b) \( F_h(x^*) = \{v \in \mathbb{R}^d: \nabla h_i(x^*)^T v > 0, \forall i \in [\ell]\} \).
• c) \( F_h(x^*) = \{v \in \mathbb{R}^d: \nabla f(x^*)^T v = 0\} \).
• d) \( F_h(x^*) = \{v \in \mathbb{R}^d: \nabla f(x^*)^T v + \sum_{i=1}^\ell \lambda_i^* \nabla h_i(x^*)^T v = 0\} \).

Which of the following is a correct statement of Taylor's Theorem (to second order) for a twice continuously differentiable function \( f: D \to \mathbb{R} \), where \( D \subseteq \mathbb{R}^d \), at an interior point \( x_0 \in D \)?

• a) For any \( x \in B_\delta(x_0) \), \( f(x) = f(x_0) + \nabla f(x_0)^T (x - x_0) + \frac{1}{2}(x - x_0)^T Hf(x_0 + \xi(x - x_0))(x - x_0) \) for some \( \xi \in (0,1) \).
• b) For any \( x \in B_\delta(x_0) \), \( f(x) = f(x_0) + \nabla f(x_0 + \xi(x - x_0))^T (x - x_0) + \frac{1}{2}(x - x_0)^T Hf(x_0 + \xi(x - x_0))(x - x_0) \).
• c) For any \( x \in B_\delta(x_0) \), \( f(x) = f(x_0) + \nabla f(x_0 + \xi(x - x_0))^T (x - x_0) \).
• d) For any \( x \in B_\delta(x_0) \), \( f(x) = f(x_0) + \frac{1}{2}(x_0 + \xi(x - x_0))^T Hf(x_0)(x_0 + \xi(x - x_0)) \).