StudyBrixFree worksheets, study guides and courses

Artificial intelligence · lesson 1 of 4

Linear regression, residuals and squared loss

A linear model predicts yhat=w·x+b. Linear refers to the parameters; feature transformations may make the relation nonlinear in original inputs.

University8 practice questionsNotes + worked examples
Take the quizStudy guideRevision sheet

Learn the ideas

Prerequisites: Algebra, averages and basic derivatives.

Prediction

A linear model predicts yhat=w·x+b. Linear refers to the parameters; feature transformations may make the relation nonlinear in original inputs.

Residual

Define residual consistently, for example yhat−y. Squared loss penalizes larger residuals more heavily.

Mean loss

MSE is the mean squared residual over the evaluated examples. It has squared target units and is not the same as mean absolute error.

Interpretation

Good fit on observed data does not establish causation or reliable extrapolation. Inspect residual patterns and the intended data range.

Worked examples

Example 1

For yhat=2x+3 and x=4, calculate the prediction.

2×4+3=11.

11

Example 2

For prediction 7 and target 5, calculate squared error.

(7−5)²=4.

4

Check your understanding

8 questions

Decimals or fractions are accepted for numeric answers. Proofs and written responses use sample answers and require human marking. This is practice, not a protected examination.

1 For yhat=2x+3 and x=4, calculate the prediction.

Show answer and explanation

11

2×4+3=11.

2 For prediction 7 and target 5, calculate squared error.

Show answer and explanation

4

(7−5)²=4.

3 Residuals are 1, −1 and 2. Find MSE.

Show answer and explanation

2

(1+1+4)/3=2.

4 What units does MSE have if the target is measured in metres?

Show answer and explanation

Square metres

Squared residuals have squared target units.

5 Which claim is not established by a low regression error alone?

Show answer and explanation

The input causes the target

Prediction and causation are different.

6 Why can a linear model perform badly when extrapolating far beyond training inputs?

Sample answer and marking guidance

The assumed relation may not hold outside the observed region and the training data may not constrain that behaviour.

Credit model assumptions and data support.

7 For loss (w−3)² at w=1, find derivative with respect to w.

Show answer and explanation

-4

The derivative is 2(w−3)=−4.

8 Describe one useful residual diagnostic.

Sample answer and marking guidance

Plot residuals against fitted values or inputs to check systematic patterns, varying spread or unusual points, then examine their causes.

Credit a concrete diagnostic and interpretation.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.