Artificial intelligence · University
Gradient descent and learning-rate choices · revision notes
Gradient descent uses θnew=θ−η∇L(θ) with a positive step size η. The negative gradient is locally downhill for a differentiable objective.
Key ideas and conditions
Update
Gradient descent uses θnew=θ−η∇L(θ) with a positive step size η. The negative gradient is locally downhill for a differentiable objective.
Learning rate
A step that is too large can overshoot or diverge. A very small step can be slow; no one value is suitable for every objective.
Batches
Full-batch and minibatch methods use different gradient estimates. Stochastic variation can affect the optimization trajectory.
Checking
A finite-difference check can help detect derivative implementation errors on small examples. It is a numerical diagnostic with step-size limitations.
Explain without looking
- What does a minibatch gradient use?
- Why should a gradient check avoid declaring success from only one point?
Further reading and review references
These are reference links, not copied exercises, institutional endorsement or proof of independent review.
AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.