StudyBrixFree worksheets, study guides and courses

Artificial intelligence · lesson 2 of 4

Backpropagation and the chain rule

A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

University8 practice questionsNotes + worked examples
Take the quizStudy guideRevision sheet

Learn the ideas

Prerequisites: Derivatives and computational graphs.

Graph

A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

Chain rule

For L depending on y and y on x, dL/dx=(dL/dy)(dy/dx). Branching paths contribute additively.

Backpropagation

Reverse-mode differentiation efficiently computes a scalar loss gradient with respect to many parameters. It is differentiation, not the optimizer update itself.

Diagnostics

Check shapes, signs and numerical derivatives on small examples. Saturation, poor scaling and depth can affect gradient propagation.

Worked examples

Example 1

If y=3x and L=y², find dL/dx at x=2.

y=6, dL/dy=12 and dy/dx=3, so the product is 36.

36

Example 2

For L=(wx−y)², x=2,y=1,w=1, find dL/dw.

2(wx−y)x=2(2−1)2=4.

4

Check your understanding

8 questions

Decimals or fractions are accepted for numeric answers. Proofs and written responses use sample answers and require human marking. This is practice, not a protected examination.

1 If y=3x and L=y², find dL/dx at x=2.

Show answer and explanation

36

y=6, dL/dy=12 and dy/dx=3, so the product is 36.

2 For L=(wx−y)², x=2,y=1,w=1, find dL/dw.

Show answer and explanation

4

2(wx−y)x=2(2−1)2=4.

3 What does backpropagation primarily compute?

Show answer and explanation

Gradients through a computational graph

An optimizer subsequently uses gradients to update parameters.

4 At a branching node, contributions from multiple downstream paths are combined how?

Show answer and explanation

They are added

The total derivative sums path contributions.

5 Two independent path contributions to a parameter gradient are 3 and −1. Find total gradient.

Show answer and explanation

2

3+(−1)=2.

6 Why should an implementation keep differentiation separate from parameter updates?

Sample answer and marking guidance

The gradient specifies local sensitivity; the update rule controls how it is used. Separation improves correctness, testing and optimizer flexibility.

Credit distinct responsibilities.

7 Which is a useful small-model verification technique?

Show answer and explanation

Compare with finite-difference gradients

A numerical derivative can reveal errors, with appropriate tolerances.

8 Explain one cause of vanishing gradients without claiming all deep models must fail.

Sample answer and marking guidance

Products of small derivative factors along many layers can reduce upstream gradients. Architecture, activations and normalization can change that behaviour.

Credit chain-rule multiplication and qualifications.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.