A dense layer has 3 inputs and 4 outputs with one bias per output. Count parameters.
Answer and explanation
16
3×4+4=16.
Subjects
Deep learning
Choose questions individually, practise online, or create a worksheet with matching answers. Counts include generated variants, not only authored originals.
Make a worksheet911 question instances
Sample exercises
Find ReLU(−2).
Answer and explanation
0
max(0,−2)=0.
For weights (2,−1), input (3,4) and bias 1, find the preactivation.
Answer and explanation
3
2×3−1×4+1=3.
What happens if several affine layers are stacked with no nonlinear activation?
- Their composition is another affine map
- They necessarily output zero
- They automatically become any possible function
- They have no parameters
Answer and explanation
Their composition is another affine map
Affine compositions remain affine.
Which dimensions must match in Wx?
- The rows of W and word count
- Nothing ever
- Only the file sizes
- The columns of W and entries of x
Answer and explanation
The columns of W and entries of x
Matrix multiplication requires compatible inner dimensions.
Why does a larger parameter count not guarantee better generalization?
Answer and explanation
Capacity, data quality, regularization, optimization and task structure all matter. More parameters alone do not establish unseen-data performance.
Credit multiple relevant factors and evaluation.
Study notes
Neural layers, activations and parameter counts
A dense layer computes z=Wx+b followed by an activation. Dimensions must agree; each output has its own weighted combination.
UniversityNeural layers, activations and parameter counts · revision notes
A dense layer computes z=Wx+b followed by an activation. Dimensions must agree; each output has its own weighted combination.
UniversityBackpropagation and the chain rule
A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.
UniversityBackpropagation and the chain rule · revision notes
A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.
UniversityAttention, softmax and transformer structure
Scaled dot-product attention forms weights softmax(QKᵀ/sqrt(dk)) and uses them to combine value vectors. Dimensions and masks are part of the definition.
Advanced universityAttention, softmax and transformer structure · revision notes
Scaled dot-product attention forms weights softmax(QKᵀ/sqrt(dk)) and uses them to combine value vectors. Dimensions and masks are part of the definition.
Advanced universityEmbeddings, similarity and retrieval representations
An embedding represents an item as a vector learned or designed for a task. Nearby vectors reflect the representation’s notion of similarity, not universal meaning.
UniversityEmbeddings, similarity and retrieval representations · revision notes
An embedding represents an item as a vector learned or designed for a task. Nearby vectors reflect the representation’s notion of similarity, not universal meaning.
UniversityParameter count in a dense layer
Assume one trainable bias per output and no parameter sharing.
UniversityParameter count in a dense layer · formula reference
Assume one trainable bias per output and no parameter sharing.
UniversityOutput of a scalar ReLU unit
This asks for a forward pass, not the derivative at the nondifferentiable point zero.
UniversityOutput of a scalar ReLU unit · formula reference
This asks for a forward pass, not the derivative at the nondifferentiable point zero.
UniversityA two-score softmax probability
Softmax normalizes scores; this alone does not prove probabilities are calibrated to real-world frequencies.
Advanced universityA two-score softmax probability · formula reference
Softmax normalizes scores; this alone does not prove probabilities are calibrated to real-world frequencies.
Advanced universityRelated courses
Artificial intelligence4 lessons
AI foundations and responsible use
Understand learning tasks, datasets, honest evaluation, privacy and human responsibility. No paid AI tool is required.
32 quiz questions · School (K–12)
Artificial intelligence4 lessons
Mathematics of machine learning
Work through regression, gradients, classification and generalization using paper-based examples.
32 quiz questions · University
Artificial intelligence4 lessons
Machine learning methods and validation
Compare trees, clustering, PCA and validation without confusing training performance with evidence of generalization.
32 quiz questions · University
Artificial intelligence4 lessons
Neural networks, attention and embeddings
Build from artificial neurons and backpropagation to attention and embedding geometry.
32 quiz questions · University, Advanced university
Artificial intelligence4 lessons
Generative AI, RAG and practical evaluation
Practise clear prompting, retrieval-grounded workflows, evidence checking and permission-aware deployment.
32 quiz questions · School (K–12), University