Production AI Checklist

Use this before calling any AI project production-style.

Product Fit

• The problem is clearly defined.
• Non-AI alternatives were considered.
• Success metrics are measurable.
• Failure impact is understood.
• Human fallback exists when needed.

Data

• Data sources are documented.
• Data quality checks exist.
• Sensitive data handling is defined.
• Retention rules are defined.
• Test data is separated from production data.

Model and Prompt

• Model choice is intentional.
• Prompt versions are tracked.
• Structured outputs are validated.
• Failure cases are tested.
• Fallback strategy exists.

Evaluation

• Golden dataset exists.
• Regression tests exist.
• Quality thresholds are defined.
• Cost and latency are measured.
• Evaluation report is documented.

Security

• Authentication exists.
• Authorization exists.
• Secrets are not hardcoded.
• Audit logs exist.
• Prompt injection risks are tested.

Operations

• Logs exist.
• Metrics exist.
• Alerts exist.
• Backup plan exists.
• Incident response notes exist.