# Linear regression, residuals and squared loss

> A linear model predicts yhat=w·x+b. Linear refers to the parameters; feature transformations may make the relation nonlinear in original inputs.

Course: [Mathematics of machine learning](https://studybrix.com/en/courses/machine-learning-mathematics/index.md)
Level: university
Prerequisites: Algebra, averages and basic derivatives.

## Notes

### Prediction

A linear model predicts yhat=w·x+b. Linear refers to the parameters; feature transformations may make the relation nonlinear in original inputs.

### Residual

Define residual consistently, for example yhat−y. Squared loss penalizes larger residuals more heavily.

### Mean loss

MSE is the mean squared residual over the evaluated examples. It has squared target units and is not the same as mean absolute error.

### Interpretation

Good fit on observed data does not establish causation or reliable extrapolation. Inspect residual patterns and the intended data range.

## Practice questions

### 1. For yhat=2x+3 and x=4, calculate the prediction.

Answer: 11

Explanation: 2×4+3=11.

### 2. For prediction 7 and target 5, calculate squared error.

Answer: 4

Explanation: (7−5)²=4.

### 3. Residuals are 1, −1 and 2. Find MSE.

Answer: 2

Explanation: (1+1+4)/3=2.

### 4. What units does MSE have if the target is measured in metres?

A. Seconds
B. Metres only
C. No units necessarily
D. Square metres

Answer: Square metres

Explanation: Squared residuals have squared target units.

### 5. Which claim is not established by a low regression error alone?

A. The input causes the target
B. The evaluated predictions fit the chosen metric
C. Residuals can be inspected
D. A baseline comparison is possible

Answer: The input causes the target

Explanation: Prediction and causation are different.

### 6. Why can a linear model perform badly when extrapolating far beyond training inputs?

Sample answer / marking guidance: The assumed relation may not hold outside the observed region and the training data may not constrain that behaviour.

Explanation: Credit model assumptions and data support.

### 7. For loss (w−3)² at w=1, find derivative with respect to w.

Answer: -4

Explanation: The derivative is 2(w−3)=−4.

### 8. Describe one useful residual diagnostic.

Sample answer / marking guidance: Plot residuals against fitted values or inputs to check systematic patterns, varying spread or unusual points, then examine their causes.

Explanation: Credit a concrete diagnostic and interpretation.

## Further reading

- https://developers.google.com/machine-learning/crash-course
- https://cs229.stanford.edu/

Original AI-assisted educational draft; not independently teacher-reviewed. References support further study, not copied exercises, affiliation or curriculum accreditation.