Project 03: Transformer and LLM Lab

Objective

Create a lab that demonstrates attention from scratch, transformer concepts, hosted LLM integration, structured outputs, tool calling, and cost comparison.

Learning Outcomes

• Implement attention mechanics.
• Understand tokenization and embeddings.
• Use LLM APIs safely.
• Validate structured output.
• Compare model cost and quality.

Tasks

• Implement scaled dot-product attention.
• Implement a tiny transformer block.
• Create tokenization examples.
• Integrate one hosted LLM.
• Create a structured output prompt.
• Validate output with a schema.
• Implement one tool/function call.
• Compare multiple prompts or models.
• Write cost and latency notes.

Deliverables

• attention_from_scratch notebook or script.
• LLM integration example.
• Structured output examples.
• Tool calling example.
• Cost comparison table.
• Lessons learned document.

Tips

• Keep the scratch implementation small and readable.
• Do not expose API keys in examples.
• Store prompt versions as files.
• Treat schema validation as mandatory for automation.