Artificial intelligence · University
Backpropagation and the chain rule · revision notes
A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.
Key ideas and conditions
Graph
A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.
Chain rule
For L depending on y and y on x, dL/dx=(dL/dy)(dy/dx). Branching paths contribute additively.
Backpropagation
Reverse-mode differentiation efficiently computes a scalar loss gradient with respect to many parameters. It is differentiation, not the optimizer update itself.
Diagnostics
Check shapes, signs and numerical derivatives on small examples. Saturation, poor scaling and depth can affect gradient propagation.
Explain without looking
- Which is a useful small-model verification technique?
- Explain one cause of vanishing gradients without claiming all deep models must fail.
Further reading and review references
These are reference links, not copied exercises, institutional endorsement or proof of independent review.
AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.