Lab: Attention From Scratch

Goal

Implement scaled dot-product attention on tiny matrices.

Tasks

• Create query, key, and value matrices.
• Compute attention scores.
• Scale scores.
• Apply softmax.
• Multiply by values.
• Explain each tensor shape.

Failure Case

• Use mismatched dimensions and explain the error.

Output

• Script or notebook.
• Shape explanation.
• Short note on why attention is useful.