Exercises: Transformers and LLMs

Concept Drills

• Explain tokenization with three examples.
• Explain embeddings with a simple analogy.
• Draw self-attention.
• Compare encoder, decoder, and encoder-decoder models.

Coding Drills

• Implement scaled dot-product attention.
• Implement multi-head attention conceptually or in code.
• Call an LLM API and validate JSON output.
• Build a tool calling example.
• Track token usage and cost.

Failure Drills

• Force invalid JSON output and handle it.
• Create an ambiguous prompt and improve it.
• Test hallucination with missing context.
• Compare deterministic and creative settings.

Output

• Attention implementation.
• LLM integration script.
• Prompt evaluation table.
• Cost comparison notes.