MMiDS 4.3: Self-Assessment Quiz

Let \( \alpha_1, \dots, \alpha_n \) be data points in \( \mathbb{R}^m \). What is the objective of the best approximating subspace problem?

• a) To find a linear subspace \( Z \) of \( \mathbb{R}^m \) that minimizes the sum of the distances between the \( \alpha_i \)'s and \( Z \).
• b) To find a linear subspace \( Z \) of \( \mathbb{R}^m \) that minimizes the sum of the squared distances between the \( \alpha_i \)'s and their orthogonal projections onto \( Z \).
• c) To find a linear subspace \( Z \) of \( \mathbb{R}^m \) that maximizes the sum of the squared norms of the orthogonal projections of the \( \alpha_i \)'s onto \( Z \).
• d) Both b and c.

Consider the data points \( \alpha_1 = (-2,2) \) and \( \alpha_2 = (3,-3) \). For \( k=1 \), what is the solution of the best approximating subspace problem?

• a) \( Z = \{(x,y) \in \mathbb{R}^2 : y = x\} \)
• b) \( Z = \{(x,y) \in \mathbb{R}^2 : y = -x\} \)
• c) \( Z = \{(x,y) \in \mathbb{R}^2 : y = x+1\} \)
• d) \( Z = \{(x,y) \in \mathbb{R}^2 : y = x-1\} \)

Let \( A \in \mathbb{R}^{n \times m} \) with rows \( \alpha_i^T \), \( i=1,\dots,n \). A solution to the best approximating subspace problem is obtained by solving:

• a) \( \max_{w_1,\dots,w_k} \sum_{j=1}^k \| Aw_j \|^2 \) over all orthonormal lists \( w_1,\dots,w_k \) of length \( k \).
• b) \( \min_{w_1,\dots,w_k} \sum_{j=1}^k \| Aw_j \|^2 \) over all orthonormal lists \( w_1,\dots,w_k \) of length \( k \).
• c) \( \max_{w_1,\dots,w_k} \sum_{j=1}^k \| Aw_j \| \) over all orthonormal lists \( w_1,\dots,w_k \) of length \( k \).
• d) \( \min_{w_1,\dots,w_k} \sum_{j=1}^k \| Aw_j \| \) over all orthonormal lists \( w_1,\dots,w_k \) of length \( k \).

What is the significance of the leading right singular vectors in the SVD?

• a) They form an orthonormal basis for the column space of \( A \).
• b) They form an orthonormal basis for the row space of \( A \).
• c) They span the subspace that best approximates the data points.
• d) They maximize the sum of the Euclidean distances from the data points.

In the singular value decomposition \( A = \sum_{j=1}^r \sigma_j u_j v_j^T \), the vectors \( u_j \) are called:

• a) Left singular vectors
• b) Right singular vectors
• c) Singular values
• d) None of the above

Which of the following is true about the SVD of a matrix \( A \)?

• a) The SVD of \( A \) is unique.
• b) The left singular vectors of \( A \) are the eigenvectors of \( A^TA \).
• c) The right singular vectors of \( A \) are the eigenvectors of \( AA^T \).
• d) Both b and c.

What is the difference between a compact SVD and a full SVD?

• a) A compact SVD only includes the non-zero singular values, while a full SVD includes all singular values, including zeros.
• b) A compact SVD has orthogonal matrices \( U \) and \( V \), while a full SVD has orthonormal matrices \( U \) and \( V \).
• c) A compact SVD is unique, while a full SVD is not.
• d) A compact SVD can be computed for any matrix, while a full SVD can only be computed for square matrices.

Let \( A = U \Sigma V^T \) be an SVD of \( A \). Which of the following is true?

• a) \( A v_i = \sigma_i u_i \) for all \( i \).
• b) \( A^T u_i = \sigma_i v_i \) for all \( i \).
• c) \( \|Av_i\| = \sigma_i \) for all \( i \).
• d) All of the above.

Which of the following is NOT a property of the matrix \( A^TA \) in the context of the SVD?

• a) \( A^T A \) is symmetric.
• b) \( A^T A \) is positive semidefinite.
• c) The eigenvectors of \( A^T A \) are the right singular vectors of \( A \).
• d) The eigenvalues of \( A^T A \) are the squares of the singular values of \( A \).

The columns of \( U \) in the compact SVD form an orthonormal basis for:

• a) \( \mathrm{col}(A) \)
• b) \( \mathrm{row}(A) \)
• c) \( \mathrm{null}(A) \)
• d) \( \mathrm{null}(A^T) \)

Let \( A \in \mathbb{R}^{n \times m} \) have compact SVD \( A = U_1 \Sigma_1 V_1^T \). To obtain a full SVD, we complete \( U_1 \) to an orthonormal basis \( U = (U_1 \ U_2) \) of \( \mathbb{R}^n \) and \( V_1 \) to an orthonormal basis \( V = (V_1 \ V_2) \) of \( \mathbb{R}^m \). Then the columns of \( U_2 \) form an orthonormal basis of:

• a) \( \mathrm{col}(A) \)
• b) \( \mathrm{row}(A) \)
• c) \( \mathrm{null}(A) \)
• d) \( \mathrm{null}(A^T) \)