MMiDS 4.6: Self-Assessment Quiz

Which of the following is NOT a property of the induced 2-norm of a matrix \(A\)?

• a) \(\|A\|_2 \ge 0\)
• b) \(\|A\|_2 = 0\) if and only if \(A = 0\)
• c) \(\|\alpha A\|_2 = |\alpha| \|A\|_2\) for all \(\alpha \in \mathbb{R}\)
• d) \(\|A + B\|_2 = \|A\|_2 + \|B\|_2\) for all matrices \(A\) and \(B\) of the same dimensions

Which of the following best describes the Frobenius norm of a matrix \(A \in \mathbb{R}^{n \times m}\)?

• a) The maximum singular value of \(A\).
• b) The square root of the sum of the squares of all entries in \(A\).
• c) The maximum absolute row sum of \(A\).
• d) The maximum absolute column sum of \(A\).

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with compact SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\), where \(\sigma_1 \geq \sigma_2 \geq \cdots \sigma_r > 0\). Which of the following is true about the induced 2-norm of \(A\)?

• a) \(\|A\|_2^2 = \sum_{j=1}^r \sigma_j^2\)
• b) \(\|A\|_2^2 = \sigma_1^2\)
• c) \(\|A\|_2^2 = \sigma_r^2\)
• d) \(\|A\|_2^2 = \prod_{j=1}^r \sigma_j^2\)

Let \(A \in \mathbb{R}^{n \times m}\) with \(m \leq n\) and let \(B \in \mathbb{R}^{n \times m}\) be a matrix of rank \(k \leq \min\{n,m\}\). Let \(B_{\perp} \in \mathbb{R}^{n \times m}\) be the matrix obtained by projecting each row of \(A\) onto the row space of \(B\). Which of the following is true?

• a) \(\|A - B_{\perp}\|_F \geq \|A - B\|_F\)
• b) \(\|A - B_{\perp}\|_F \leq \|A - B\|_F\)
• c) \(\|A - B_{\perp}\|_F = \|A - B\|_F\)
• d) The relationship between \(\|A - B_{\perp}\|_F\) and \(\|A - B\|_F\) cannot be determined.

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) and let \(A_k = \sum_{j=1}^k \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) be the truncated SVD with \(k < r\). Which of the following is true about the Frobenius norm of \(A - A_k\)?

• a) \(\|A - A_k\|_F^2 = \sum_{j=1}^k \sigma_j^2\)
• b) \(\|A - A_k\|_F^2 = \sum_{j=k+1}^r \sigma_j^2\)
• c) \(\|A - A_k\|_F^2 = \sigma_k^2\)
• d) \(\|A - A_k\|_F^2 = \sigma_{k+1}^2\)

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) and let \(A_k = \sum_{j=1}^k \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) be the truncated SVD with \(k < r\). Which of the following is true about the induced 2-norm of \(A - A_k\)?

• a) \(\|A - A_k\|_2^2 = \sum_{j=1}^k \sigma_j^2\)
• b) \(\|A - A_k\|_2^2 = \sum_{j=k+1}^r \sigma_j^2\)
• c) \(\|A - A_k\|_2^2 = \sigma_k^2\)
• d) \(\|A - A_k\|_2^2 = \sigma_{k+1}^2\)

What is the pseudoinverse of an \(n \times m\) matrix \(A\) with compact SVD \(A = \sum_{l=1}^r \sigma_l \mathbf{u}_l \mathbf{v}_l^T\)? [Uses Section 4.8.2.2.]

• a) \(A^+ = \sum_{l=1}^r \sigma_l^{-1} \mathbf{u}_l \mathbf{v}_l^T\)
• b) \(A^+ = \sum_{l=1}^r \sigma_l \mathbf{u}_l \mathbf{v}_l^T\)
• c) \(A^+ = \sum_{l=1}^r \sigma_l^{-1} \mathbf{v}_l \mathbf{u}_l^T\)
• d) \(A^+ = \sum_{l=1}^r \sigma_l \mathbf{v}_l \mathbf{u}_l^T\)

Let \(A\) be an \(n \times m\) matrix with \(m \le n\). Which of the following is true about the pseudoinverse \(A^+\)? [Uses Section 4.8.2.2.]

• a) \(AA^+ = I_{n \times n}\) if \(A\) has full column rank.
• b) \(A^+A = I_{m \times m}\) if \(A\) has full row rank.
• c) \(A^+ = A^{-1}\) if \(A\) is a square, nonsingular matrix.
• d) All of the above.

Let \(A \in \mathbb{R}^{n \times m}\) with \(m \leq n\) and assume that \(A\) has full column rank \(m\). Which of the following is true about the pseudoinverse \(A^+\)? [Uses Section 4.8.2.2.]

• a) \(A^+ = (A^T A)^{-1} A^T\)
• b) \(A^+ = A^T (A A^T)^{-1}\)
• c) \(A^+ = (A A^T)^{-1} A^T\)
• d) \(A^+ = A^T (A^T A)^{-1}\)

Let \(A \in \mathbb{R}^{n \times m}\) with \(m > n\) and assume that \(A\) has full row rank \(n\). Which of the following is true about the pseudoinverse \(A^+\)? [Uses Section 4.8.2.2.]

• a) \(A^+ = (A^T A)^{-1} A^T\)
• b) \(A^+ = A^T (A A^T)^{-1}\)
• c) \(A^+ = (A A^T)^{-1} A^T\)
• d) \(A^+ = A^T (A^T A)^{-1}\)

Let \(A\) be an \(n \times m\) matrix with \(m > n\) and full row rank. Which of the following is true about the solution \(\mathbf{x}^* = A^+ \mathbf{b}\) to the system \(A \mathbf{x} = \mathbf{b}\)?

• a) It is the unique solution.
• b) It is the solution with the smallest 2-norm.
• c) It is the solution with the largest 2-norm.
• d) None of the above.

What is the purpose of ridge regression?

• a) To find the best low-rank approximation of a matrix.
• b) To compute the pseudoinverse of a matrix.
• c) To address multicollinearity in overdetermined linear systems.
• d) To solve underdetermined linear systems.

The ridge regression problem is formulated as \(\min_{x \in \mathbb{R}^m} \|A\mathbf{x} - \mathbf{b}\|_2^2 + \lambda \|\mathbf{x}\|_2^2\). What is the role of the parameter \(\lambda\)?

• a) It controls the trade-off between fitting the data and minimizing the norm of the solution.
• b) It determines the rank of the matrix \(A\).
• c) It is the smallest singular value of \(A\).
• d) It is the largest singular value of \(A\).

Let \(A\) be an \(n \times m\) matrix with compact SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\). How does the ridge regression solution \(\mathbf{x}^{**}\) compare to the least squares solution \(\mathbf{x}^*\)?

• a) \(\mathbf{x}^{**}\) has larger components along the left singular vectors corresponding to small singular values.
• b) \(\mathbf{x}^{**}\) has smaller components along the left singular vectors corresponding to small singular values.
• c) \(\mathbf{x}^{**}\) is identical to \(\mathbf{x}^*\).
• d) None of the above.

Let \(A \in \mathbb{R}^{n \times m}\) with \(n \geq m\) and \(\mathbf{b} \in \mathbb{R}^n\). Consider the ridge regression problem

\[ \min_{\mathbf{x} \in \mathbb{R}^m} \|A \mathbf{x} - \mathbf{b}\|_2^2 + \lambda \|\mathbf{x}\|_2^2, \]

where \(\lambda > 0\). Which of the following is the solution to this problem?

• a) \(\mathbf{x}^{**} = (A^T A)^{-1} A^T \mathbf{b}\)
• b) \(\mathbf{x}^{**} = (A^T A + \lambda I_{m \times m})^{-1} A^T \mathbf{b}\)
• c) \(\mathbf{x}^{**} = (A A^T + \lambda I_{n \times n})^{-1} A^T \mathbf{b}\)
• d) \(\mathbf{x}^{**} = (A A^T)^{-1} A^T \mathbf{b}\)

Let \(A \in \mathbb{R}^{n \times n}\) be a square nonsingular matrix with compact SVD \(A = \sum_{j=1}^n \sigma_j \mathbf{u}_j \mathbf{v}_j^T\). Which of the following is true about the induced 2-norm of the inverse \(A^{-1}\)?

• a) \(\|A^{-1}\|_2 = \sigma_1\)
• b) \(\|A^{-1}\|_2 = \sigma_n\)
• c) \(\|A^{-1}\|_2 = \sigma_1^{-1}\)
• d) \(\|A^{-1}\|_2 = \sigma_n^{-1}\)