A first production model is rarely blocked by the model. It is blocked by the surrounding facts: nobody can say where a training column came from, the evaluation set changed without notice, or a single engineer holds the only working copy of the preprocessing script. This checklist is a way to find those gaps while they are still cheap to close.

Why a Readiness Review Pays Off

Prototypes hide fragility. A notebook can read a file from a laptop, apply ad hoc fixes, and produce a convincing demo. Production asks harder questions: can the result be reproduced next month, can another person run it, and what happens when the upstream data changes shape? A short readiness review turns those questions into concrete work items before launch rather than incidents after it.

The Five Areas to Examine

1. Data access and ownership

Every dataset feeding the system should have a named owner, a documented location, and a clear access path that does not depend on one person’s credentials. If a source is a manual export, treat that as a risk to be automated, not a permanent arrangement.

2. Data quality and contracts

Decide what “good” means for each input: expected columns, types, ranges, null behavior, and freshness. Write those expectations down as checks that run automatically. A schema change from an upstream team should fail loudly in your pipeline, not silently shift a model’s behavior.

3. Reproducibility

You should be able to name the exact data, code, and configuration behind any model that reaches production. That usually means versioned datasets or snapshots, pinned dependencies, and recorded parameters. If you cannot rebuild last month’s model, you cannot confidently debug this month’s.

4. Pipeline operations

Pipelines need scheduling, retries, alerting, and a defined response when they fail. Decide what happens to the serving system if a data refresh is late: does it serve stale results, degrade gracefully, or stop?

5. Evaluation and monitoring

Hold out evaluation data that is stable and representative, and plan how you will notice drift after launch. Monitoring that only watches infrastructure health will miss a model that is running fine technically but producing worse results.

The Checklist

Area	Question to answer	Evidence of readiness
Ownership	Who owns each source dataset?	Named owner and contact per source
Access	Can the pipeline run without personal credentials?	Service identity with scoped permissions
Quality	Are input expectations enforced automatically?	Validation checks that block bad loads
Reproducibility	Can we rebuild a past model?	Versioned data, code, and config recorded
Operations	What happens if a refresh fails?	Alerting and documented fallback behavior
Evaluation	Is the test set stable and representative?	Frozen evaluation set with documented origin
Monitoring	How will we detect drift or degradation?	Input and output monitoring with owners
Security	Is sensitive data classified and protected?	Access controls and a recorded review

Use the right-hand column as a gate. If you cannot point to evidence, treat the row as open.

A Hypothetical Example

Imagine a team preparing to launch a demand-forecasting model. In review, they discover that one input table is refreshed by a script on an engineer’s workstation, that the evaluation set is simply the most recent month of data and therefore changes every run, and that no alert exists if the refresh stops. None of these are modeling problems. The team moves the refresh to a scheduled job with a service identity, freezes a labeled evaluation set with a written description of how it was drawn, and adds a freshness check that pages the on-call owner. Launch slips by a short period, but the first incident that would have occurred never happens.

Gaps That Show Up Most Often

• Hidden manual steps. A spreadsheet edit, a one-off SQL fix, or a hand-run script that nobody documented.
• Training data that includes the future. Features computed with information that would not be available at prediction time, which inflates offline results.
• No separation of environments. Experiments writing to the same tables that production reads.
• Unclear rollback. No plan for reverting to a previous model or dataset version if results degrade.
• Security as an afterthought. Sensitive fields flowing into training data without anyone having decided they should.

Making the Review Lightweight

You do not need a heavyweight process. A one-hour working session with the person who built the prototype, someone who will operate it, and a data owner is usually enough. Walk the checklist, mark each row as ready, open, or accepted risk, and assign an owner and a date to every open item. The “accepted risk” category matters: some gaps are fine for a low-stakes internal tool and unacceptable for a customer-facing feature, and writing that decision down keeps it deliberate.

Frequently Asked Questions

Is this checklist only for large organizations? No. Small teams benefit most because they have the fewest people to absorb a surprise. Scale the depth of each check to the stakes of the system.

Do LLM applications need the same readiness review? Mostly yes. You still need data ownership, quality checks on source documents, evaluation sets, and monitoring. The artifacts differ, such as prompt versions and retrieval corpora, but the principles carry over.

What if we cannot close every gap before launch? That is normal. Record which gaps are accepted, why, and when they will be revisited, and make sure someone with authority agrees to the risk.

How do we handle compliance requirements? Requirements vary by jurisdiction and industry, so involve your legal, privacy, or compliance professionals early rather than treating this checklist as a substitute for their guidance.

Who should own the checklist after launch? Whoever operates the system. Re-run it whenever data sources, models, or traffic patterns change substantially.

Next Step

Copy the table into your project notes, schedule a one-hour review with your builder, operator, and a data owner, and assign a named owner and date to every row that is not yet backed by evidence.