The distance between an impressive AI demo and a shipped enterprise feature is not model quality. It's a stack of boring requirements that nobody puts in the keynote: security review, data handling, evals, fallbacks, and a cost line someone signs off on. Here's the checklist I wish I'd had.
1. Decide what the model is allowed to see
Before the first prompt is written, write the data contract: which fields leave your boundary, which are redacted, which never leave the database. Security review will ask; having the answer as a one-page table turns a six-week review into a one-week one. Log the actual payloads in staging and diff them against the contract, because prompts accrete fields the way closets accrete coats.
2. Evals before features
An eval set of even fifty real examples with expected outputs changes everything: model upgrades become an afternoon instead of a leap of faith, prompt changes get a regression gate, and "is it better?" becomes a number. Build the eval harness the same week you build the prototype. Retrofitting evals onto a shipped feature is archaeology.
3. Design the failure, not just the feature
The LLM will time out, return malformed JSON, and occasionally be confidently wrong. Each needs a decision, not a hope:
- Timeout: degrade to the non-AI path, visibly and fast.
- Malformed output: validate against a schema, retry once, then degrade.
- Wrong-but-plausible: this is the dangerous one. Anywhere the output drives an action, keep a human approval step until the eval numbers earn its removal.
4. Put a ceiling on cost per unit of work
Token costs don't fail loudly; they compound quietly. Give every feature a cost budget per request and an alert when the rolling average crosses it. Cache aggressively: system prompts, retrieved context, repeated questions. In most enterprise workloads, half the spend is answering the same question twice.
5. Make it auditable
When a user asks "why did it say that?", you need the prompt, the context, the model version, and the output, joined by a request ID. This is table stakes for enterprise trust and takes an afternoon to build on day one. It also turns every production complaint into a reproducible test case, which quietly feeds requirement #2.
6. Ship narrow, then widen
The features that survive are the ones that started embarrassingly narrow: one document type, one team, one workflow. Narrow scope makes evals tractable, failure modes enumerable, and wins legible. The demo that does everything is the feature that ships nothing.
None of this is glamorous. All of it is why the feature is still running a year later.