Failure Cases

Use this bank to test systems beyond happy paths.

ML Failures

• Data leakage produces unrealistic validation scores.
• Training data distribution differs from test data.
• Missing values appear in production inputs.
• Class imbalance hides poor minority-class performance.
• Model performs well on average but fails on important segments.

LLM Failures

• Model returns invalid JSON.
• Model invents facts.
• Model ignores output format.
• Model refuses valid requests because the prompt is unclear.
• Model gives different answers for equivalent prompts.

RAG Failures

• Correct document is not retrieved.
• Retrieved context is irrelevant.
• Answer is correct but citation is wrong.
• Answer is unsupported by context.
• Restricted document is retrieved for unauthorized user.
• Malicious document instruction affects the answer.

Agent Failures

• Agent chooses wrong tool.
• Agent loops repeatedly.
• Agent executes write action without confirmation.
• Agent retries and creates duplicate records.
• Agent reveals tool or system internals.
• Agent attempts unauthorized action.

Cloud and Operations Failures

• API latency exceeds target.
• Database connection pool is exhausted.
• Model provider is unavailable.
• Logs do not contain enough debugging context.
• Cost spikes unexpectedly.
• Backup restore process is untested.

Security Failures

• Secret appears in logs.
• User can access another tenant's data.
• Prompt injection bypasses intended behavior.
• Tool input validation accepts unsafe data.
• Audit log misses critical action details.

Failure Case Template

• Name:
• System area:
• Input:
• Expected safe behavior:
• Actual behavior:
• Severity:
• Fix:
• Regression test: