MMiDS 4.6: Self-Assessment Quiz

Which of the following is NOT a property of the induced 2-norm of a matrix \(A\)?

a) \(\|A\|_2 \ge 0\)
b) \(\|A\|_2 = 0\) if and only if \(A = 0\)
c) \(\|\alpha A\|_2 = |\alpha| \|A\|_2\) for all \(\alpha \in \mathbb{R}\)
d) \(\|A + B\|_2 = \|A\|_2 + \|B\|_2\) for all matrices \(A\) and \(B\) of the same dimensions

Which of the following best describes the Frobenius norm of a matrix \(A \in \mathbb{R}^{n \times m}\)?

a) The maximum singular value of \(A\).
b) The square root of the sum of the squares of all entries in \(A\).
c) The maximum absolute row sum of \(A\).
d) The maximum absolute column sum of \(A\).

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with compact SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\), where \(\sigma_1 \geq \sigma_2 \geq \cdots \sigma_r > 0\). Which of the following is true about the induced 2-norm of \(A\)?

a) \(\|A\|_2^2 = \sum_{j=1}^r \sigma_j^2\)
b) \(\|A\|_2^2 = \sigma_1^2\)
c) \(\|A\|_2^2 = \sigma_r^2\)
d) \(\|A\|_2^2 = \prod_{j=1}^r \sigma_j^2\)

*Let \(A \in \mathbb{R}^{n \times m}\) with \(m \leq n\) and let \(B \in \mathbb{R}^{n \times m}\) be a matrix of rank \(k \leq \min\{n,m\}\). Let \(B_{\perp} \in \mathbb{R}^{n \times m}\) be the matrix obtained by projecting each row of \(A\) onto the row space of \(B\). Which of the following is true?

a) \(\|A - B_{\perp}\|_F \geq \|A - B\|_F\)
b) \(\|A - B_{\perp}\|_F \leq \|A - B\|_F\)
c) \(\|A - B_{\perp}\|_F = \|A - B\|_F\)
d) The relationship between \(\|A - B_{\perp}\|_F\) and \(\|A - B\|_F\) cannot be determined.

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) and let \(A_k = \sum_{j=1}^k \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) be the truncated SVD with \(k < r\). Which of the following is true about the Frobenius norm of \(A - A_k\)?

a) \(\|A - A_k\|_F^2 = \sum_{j=1}^k \sigma_j^2\)
b) \(\|A - A_k\|_F^2 = \sum_{j=k+1}^r \sigma_j^2\)
c) \(\|A - A_k\|_F^2 = \sigma_k^2\)
d) \(\|A - A_k\|_F^2 = \sigma_{k+1}^2\)

Let \(A \in \mathbb{R}^{n \times m}\) be a matrix with SVD \(A = \sum_{j=1}^r \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) and let \(A_k = \sum_{j=1}^k \sigma_j \mathbf{u}_j \mathbf{v}_j^T\) be the truncated SVD with \(k < r\). Which of the following is true about the induced 2-norm of \(A - A_k\)?

a) \(\|A - A_k\|_2^2 = \sum_{j=1}^k \sigma_j^2\)
b) \(\|A - A_k\|_2^2 = \sum_{j=k+1}^r \sigma_j^2\)
c) \(\|A - A_k\|_2^2 = \sigma_k^2\)
d) \(\|A - A_k\|_2^2 = \sigma_{k+1}^2\)

What is the pseudoinverse of an \(n \times m\) matrix \(A\) with compact SVD \(A = \sum_{l=1}^r \sigma_l u_l v_l^T\)?

a) \(A^+ = \sum_{l=1}^r \sigma_l^{-1} u_l v_l^T\)
b) \(A^+ = \sum_{l=1}^r \sigma_l u_l v_l^T\)
c) \(A^+ = \sum_{l=1}^r \sigma_l^{-1} v_l u_l^T\)
d) \(A^+ = \sum_{l=1}^r \sigma_l v_l u_l^T\)

Let \(A\) be an \(n \times m\) matrix with \(m \le n\). Which of the following is true about the pseudoinverse \(A^+\)?

a) \(AA^+ = I_{n \times n}\) if \(A\) has full column rank.
b) \(A^+A = I_{m \times m}\) if \(A\) has full row rank.
c) \(A^+ = A^{-1}\) if \(A\) is a square, nonsingular matrix.
d) All of the above.

Let \(A \in \mathbb{R}^{n \times m}\) with \(m \leq n\) and assume that \(A\) has full column rank \(m\). Which of the following is true about the pseudoinverse \(A^+\)?

a) \(A^+ = (A^T A)^{-1} A^T\)
b) \(A^+ = A^T (A A^T)^{-1}\)
c) \(A^+ = (A A^T)^{-1} A^T\)
d) \(A^+ = A^T (A^T A)^{-1}\)

Let \(A \in \mathbb{R}^{n \times m}\) with \(m > n\) and assume that \(A\) has full row rank \(n\). Which of the following is true about the pseudoinverse \(A^+\)?

a) \(A^+ = (A^T A)^{-1} A^T\)
b) \(A^+ = A^T (A A^T)^{-1}\)
c) \(A^+ = (A A^T)^{-1} A^T\)
d) \(A^+ = A^T (A^T A)^{-1}\)

Let \(A\) be an \(n \times m\) matrix with \(m > n\) and full row rank. Which of the following is true about the solution \(x^* = A^+b\) to the system \(Ax = b\)?

a) It is the unique solution.
b) It is the solution with the smallest 2-norm.
c) It is the solution with the largest 2-norm.
d) None of the above.

What is the purpose of ridge regression?

a) To find the best low-rank approximation of a matrix.
b) To compute the pseudoinverse of a matrix.
c) To address multicollinearity in overdetermined linear systems.
d) To solve underdetermined linear systems.

The ridge regression problem is formulated as \(\min_{x \in \mathbb{R}^m} \|Ax - b\|_2^2 + \lambda \|x\|_2^2\). What is the role of the parameter \(\lambda\)?

a) It controls the trade-off between fitting the data and minimizing the norm of the solution.
b) It determines the rank of the matrix \(A\).
c) It is the smallest singular value of \(A\).
d) It is the largest singular value of \(A\).

Let \(A\) be an \(n \times m\) matrix with compact SVD \(A = \sum_{j=1}^r \sigma_j u_j v_j^T\). How does the ridge regression solution \(x^{**}\) compare to the least squares solution \(x^*\)?

a) \(x^{**}\) has larger components along the left singular vectors corresponding to small singular values.
b) \(x^{**}\) has smaller components along the left singular vectors corresponding to small singular values.
c) \(x^{**}\) is identical to \(x^*\).
d) None of the above.

Let \(A \in \mathbb{R}^{n \times m}\) with \(n \geq m\) and \(\mathbf{b} \in \mathbb{R}^n\). Consider the ridge regression problem

\[ \min_{\mathbf{x} \in \mathbb{R}^m} \|A \mathbf{x} - \mathbf{b}\|_2^2 + \lambda \|\mathbf{x}\|_2^2, \]

where \(\lambda > 0\). Which of the following is the solution to this problem?

a) \(\mathbf{x}^{**} = (A^T A)^{-1} A^T \mathbf{b}\)
b) \(\mathbf{x}^{**} = (A^T A + \lambda I_{m \times m})^{-1} A^T \mathbf{b}\)
c) \(\mathbf{x}^{**} = (A A^T + \lambda I_{n \times n})^{-1} A^T \mathbf{b}\)
d) \(\mathbf{x}^{**} = (A A^T)^{-1} A^T \mathbf{b}\)

Let \(A \in \mathbb{R}^{n \times n}\) be a square nonsingular matrix with compact SVD \(A = \sum_{j=1}^n \sigma_j \mathbf{u}_j \mathbf{v}_j^T\). Which of the following is true about the induced 2-norm of the inverse \(A^{-1}\)?

a) \(\|A^{-1}\|_2 = \sigma_1\)
b) \(\|A^{-1}\|_2 = \sigma_n\)
c) \(\|A^{-1}\|_2 = \sigma_1^{-1}\)
d) \(\|A^{-1}\|_2 = \sigma_n^{-1}\)