MMiDS 2.5: Self-Assessment Quiz

In linear regression, the goal is to find coefficients \( \beta_j \)'s that minimize which of the following criteria?

• a) \( \sum_{i=1}^n (y_i - \beta_0 - \sum_{j=1}^d \beta_j x_{ij}) \)
• b) \( \sum_{i=1}^n (y_i - \beta_0 - \sum_{j=1}^d \beta_j x_{ij})^2 \)
• c) \( \sum_{i=1}^n (y_i - \{\beta_0 + \sum_{j=1}^d \beta_j x_{ij}\}^2) \)
• d) \( \sum_{i=1}^n |y_i - \beta_0 - \sum_{j=1}^d \beta_j x_{ij}| \)

How does the least squares method solve for the coefficients in matrix form?

• a) By finding the inverse of the data matrix.
• b) By solving the normal equations.
• c) By applying gradient descent on \( A \).
• d) By using Bayesian inference.

The normal equations for linear regression are:

• a) \( A^T A \beta = A^T y \)
• b) \( A A^T \beta = A y \)
• c) \( A^T A \beta = A y \)
• d) \( A A^T \beta = A^T y \)

In the numerical example with a degree-20 polynomial fit, the fitted curve:

• a) Fits the data perfectly.
• b) Fails to capture the overall trend in the data.
• c) Captures the noise in the data as if it were the underlying structure.
• d) Is a straight line.

Which of the following best describes overfitting?

• a) The model fits the training data well but generalizes poorly to new data.
• b) The model fits both the training data and new data well.
• c) The model fits the training data poorly but generalizes well to new data.
• d) The model ignores random noise in the training data.

What is the primary advantage of using simulated data to test the least squares method?

• a) Simulated data eliminates the need for real-world data.
• b) Simulated data provides a perfect fit without noise.
• c) Simulated data allows us to know the ground truth.
• d) Simulated data reduces computational complexity.