# Backpropagation and the chain rule

> A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

Course: [Neural networks, attention and embeddings](https://studybrix.com/en/courses/deep-learning/index.md)
Level: university
Prerequisites: Derivatives and computational graphs.

## Notes

### Graph

A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

### Chain rule

For L depending on y and y on x, dL/dx=(dL/dy)(dy/dx). Branching paths contribute additively.

### Backpropagation

Reverse-mode differentiation efficiently computes a scalar loss gradient with respect to many parameters. It is differentiation, not the optimizer update itself.

### Diagnostics

Check shapes, signs and numerical derivatives on small examples. Saturation, poor scaling and depth can affect gradient propagation.

## Practice questions

### 1. If y=3x and L=y², find dL/dx at x=2.

Answer: 36

Explanation: y=6, dL/dy=12 and dy/dx=3, so the product is 36.

### 2. For L=(wx−y)², x=2,y=1,w=1, find dL/dw.

Answer: 4

Explanation: 2(wx−y)x=2(2−1)2=4.

### 3. What does backpropagation primarily compute?

A. Only a random architecture
B. Gradients through a computational graph
C. New labels from nothing
D. A guarantee of a global minimum

Answer: Gradients through a computational graph

Explanation: An optimizer subsequently uses gradients to update parameters.

### 4. At a branching node, contributions from multiple downstream paths are combined how?

A. They are added
B. They are always multiplied together
C. They are discarded
D. Only the first is kept always

Answer: They are added

Explanation: The total derivative sums path contributions.

### 5. Two independent path contributions to a parameter gradient are 3 and −1. Find total gradient.

Answer: 2

Explanation: 3+(−1)=2.

### 6. Why should an implementation keep differentiation separate from parameter updates?

Sample answer / marking guidance: The gradient specifies local sensitivity; the update rule controls how it is used. Separation improves correctness, testing and optimizer flexibility.

Explanation: Credit distinct responsibilities.

### 7. Which is a useful small-model verification technique?

A. Ignore signs
B. Compare with finite-difference gradients
C. Use only training accuracy
D. Rename tensors

Answer: Compare with finite-difference gradients

Explanation: A numerical derivative can reveal errors, with appropriate tolerances.

### 8. Explain one cause of vanishing gradients without claiming all deep models must fail.

Sample answer / marking guidance: Products of small derivative factors along many layers can reduce upstream gradients. Architecture, activations and normalization can change that behaviour.

Explanation: Credit chain-rule multiplication and qualifications.

## Further reading

- https://developers.google.com/machine-learning/crash-course
- https://cs229.stanford.edu/

Original AI-assisted educational draft; not independently teacher-reviewed. References support further study, not copied exercises, affiliation or curriculum accreditation.