Artificial intelligence · University
Neural layers, activations and parameter counts · revision notes
A dense layer computes z=Wx+b followed by an activation. Dimensions must agree; each output has its own weighted combination.
Key ideas and conditions
Layer
A dense layer computes z=Wx+b followed by an activation. Dimensions must agree; each output has its own weighted combination.
Nonlinearity
Nonlinear activations allow compositions that are not just one affine map. Stacking affine layers alone collapses to another affine map.
Parameters
For d inputs and h outputs, a dense layer with one bias per output has dh+h parameters.
ReLU
ReLU(z)=max(0,z). It is piecewise linear and nondifferentiable at zero; implementations choose a convention there.
Explain without looking
- A layer has 5 inputs, 2 outputs and no biases. Count weights.
- Explain why saying ReLU has derivative one everywhere is incorrect.
Further reading and review references
These are reference links, not copied exercises, institutional endorsement or proof of independent review.
AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.