Neural networks, attention and embeddings

Your learning path

1. 01

   Neural layers, activations and parameter counts

   A dense layer computes z=Wx+b followed by an activation. Dimensions must agree; each output has its own weighted combination.

   University · Notes · 8 questions
2. 02

   Backpropagation and the chain rule

   A computational graph records how intermediate values compose. Forward evaluation computes values; reverse differentiation accumulates gradients.

   University · Notes · 8 questions
3. 03

   Attention, softmax and transformer structure

   Scaled dot-product attention forms weights softmax(QKᵀ/sqrt(dk)) and uses them to combine value vectors. Dimensions and masks are part of the definition.

   Advanced university · Notes · 8 questions
4. 04

   Embeddings, similarity and retrieval representations

   An embedding represents an item as a vector learned or designed for a task. Nearby vectors reflect the representation’s notion of similarity, not universal meaning.

   University · Notes · 8 questions