StudyBrixFree worksheets, study guides and courses

Artificial intelligence · University

Backpropagation and the chain rule

A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

Make a worksheet

Prerequisites

Derivatives and computational graphs.

Study notes

Graph

A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

Chain rule

For L depending on y and y on x, dL/dx=(dL/dy)(dy/dx). Branching paths contribute additively.

Backpropagation

Reverse-mode differentiation efficiently computes a scalar loss gradient with respect to many parameters. It is differentiation, not the optimizer update itself.

Diagnostics

Check shapes, signs and numerical derivatives on small examples. Saturation, poor scaling and depth can affect gradient propagation.

Worked examples

If y=3x and L=y², find dL/dx at x=2.

y=6, dL/dy=12 and dy/dx=3, so the product is 36.

36

For L=(wx−y)², x=2,y=1,w=1, find dL/dw.

2(wx−y)x=2(2−1)2=4.

4

Check your understanding

  1. What does backpropagation primarily compute?

    1. Only a random architecture
    2. Gradients through a computational graph
    3. New labels from nothing
    4. A guarantee of a global minimum

  2. At a branching node, contributions from multiple downstream paths are combined how?

    1. They are added
    2. They are always multiplied together
    3. They are discarded
    4. Only the first is kept always

  3. Two independent path contributions to a parameter gradient are 3 and −1. Find total gradient.

  4. Why should an implementation keep differentiation separate from parameter updates?

Answers and marking guidance

Answers and marking notes
  1. Gradients through a computational graph

    An optimizer subsequently uses gradients to update parameters.

  2. They are added

    The total derivative sums path contributions.

  3. 2

    3+(−1)=2.

  4. The gradient specifies local sensitivity; the update rule controls how it is used. Separation improves correctness, testing and optimizer flexibility.

    Credit distinct responsibilities.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.

AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.