AI proof of concept failure starts after the demo

An AI proof of concept usually fails after the team has proved that the model can do the interesting part. The demo answers, "Can a model produce a convincing result on selected inputs?" Production asks whether the company can supply lawful data, detect bad results, pay for every attempt, recover from failures, and put one person on the hook. Those are different tests.

I have watched founders celebrate a polished Friday demo and then spend a quarter discovering that nobody owns the permissions, the acceptance threshold, or the monthly bill. The model gets blamed because it is visible. The missing operating system around the model caused the failure.

The cure is not a longer experiment or a more impressive model. Treat the proof of concept as the first thin slice of a production service. Give it production-shaped data access, measurable acceptance rules, an explicit cost ceiling, and a named owner before the demo earns permission to expand.

A demo proves possibility, not operability

A useful demo proves one narrow technical hypothesis; it does not prove that the workflow can run safely and economically every day. Teams get into trouble when they quietly change the claim after the applause. "The model summarized twelve handpicked contracts" becomes "we proved automated contract review." They have not proved the second statement.

The distinction matters because a demonstration removes the hard edges on purpose. An engineer cleans the input, retries a timeout, chooses a strong example, and explains an odd answer in the room. A production user arrives with a scanned document, missing fields, an unfamiliar language, and no patience for a manual rescue. The demo operator absorbs variance that the future service must handle itself.

Write the proof claim before building. A good claim has an input population, a task, a quality threshold, and a constraint. For example: "For English supplier contracts received in the last twelve months, the system extracts the five approved fields with the agreed field-level accuracy, returns a result within two minutes, and costs less than $0.40 per processed document." The numbers must come from the business, not from whatever the model happened to achieve.

That sentence prevents three common substitutions. A fluent answer is not a correct answer. Average quality is not safe quality for a high-consequence field. A cheap model call is not a cheap completed workflow when people must review most outputs.

I use a short promotion record at the end of a proof. It is deliberately plain:

hypothesis: "extract approved fields from English supplier contracts"
input_population: "contracts received in the last 12 months"
acceptance:
  field_accuracy: ">= 0.97"
  p95_latency_seconds: "<= 120"
  cost_per_accepted_document_usd: "<= 0.40"
stop_conditions:
  - "restricted data reaches an unapproved processor"
  - "critical field accuracy falls below threshold"
owner: "Head of Procurement Operations"
decision: "hold"

The decision stays hold until evidence fills every line. A persuasive screen recording cannot replace that record. If the team cannot state what passed, it has a prototype, not a production candidate.

Data access must resemble the live path

The proof should use the smallest representative dataset that can expose production access problems, under permissions that resemble the live system. Synthetic samples help an engineer start, but they hide the conditions that stop deployment: customer isolation, retention limits, regional boundaries, deleted records, malformed files, and fields that the model provider may not receive.

Teams often say, "We already have the data." Usually they mean somebody can export it. Access is a technical capability; authorization is permission for this specific purpose. A database credential settles the first question and says nothing about the second. Confusing them can turn a good model experiment into a security and compliance exception that nobody will sign.

Map the live data path before choosing the model. Name the source, classification, allowed fields, transformation, processor, retention period, output store, and deletion path. If one cell says "TBD," the proof has found work rather than removed it.

Use five checks for that map:

• At the source, specify which records and tenants may enter, then attach the approved query or access policy.
• During preparation, name the fields that must be removed or masked and test the transformation with fixed examples.
• At the model call, record which processor and region may receive the minimized payload.
• For the output, set who can read results, where they live, and when the service removes them.
• For deletion, trace one source request through caches, generated results, logs, and evaluation copies.

Run the proof through the actual service boundary early. Do not let a notebook read a founder's local export for six weeks. Put a narrow API or job around it, authenticate that path, log request identifiers, and exercise it with representative records. This work feels slower during week one and saves the project from a much more expensive surprise after approval.

Production-shaped does not mean copying the whole customer database into an experiment. Start with a sampled, minimized, access-controlled set. Include ordinary cases, ugly cases, and records that must be rejected. The rejection behavior often reveals more about readiness than the model's best response.

Ask the deletion question while the dataset is small: if a customer asks you to remove a source record tomorrow, can you identify its prompts, cached content, embeddings, generated output, logs, and evaluation copies? If the answer depends on searching several personal folders, the architecture already has debt.

Evals need failure classes and business thresholds

An eval must tell the team whether the system is safe to promote, what regressed, and which failure requires action. A single average score cannot do that. It combines harmless wording differences with failures that cost money, violate policy, or send a person the wrong decision.

Start with failure classes, not a favorite metric. For a support reply assistant, the classes might include an invented policy, missed cancellation intent, exposed personal data, wrong language, unsupported promise, and merely awkward prose. The first five need separate thresholds because their consequences differ. Style can tolerate variation that privacy cannot.

Build the eval set from the stated input population. Include normal traffic, known edge cases, adversarial or confusing inputs, and examples from every segment that changes the business consequence. Keep a locked regression set that developers cannot tune against, plus a growing investigation set for failures found in testing and live use.

Each case should carry the expected behavior and the reason it matters. A compact record is enough:

{"case_id":"cancel_017","input_ref":"fixture://cancel_017","expected":{"action":"escalate","must_not_include":["refund approved"]},"segment":"regulated_account","severity":"critical"}

That structure lets a deterministic check verify the forbidden promise while a reviewer judges tone separately. Use code for exact fields, formats, policy phrases, calculations, and tool calls. Use model-based grading only where semantic judgment is unavoidable, and calibrate that grader against decisions made by people. A grader that agrees with itself proves very little.

NIST's AI Risk Management Framework separates measurement from management. That is a useful correction to teams that treat an eval dashboard as the decision. The score is evidence. A named owner still has to decide whether the remaining risk fits the workflow and what control will contain it.

Set thresholds before the final run. If the team chooses the pass line after seeing an 89 percent result, it will discover a business reason that 88 percent is acceptable. Thresholds should come from the cost of false acceptance, false rejection, and human review. Some workflows need abstention: the system should say "I cannot decide" and route the case rather than produce its best guess.

The release report should show results by failure class and important segment, not only in aggregate. A system can improve overall while getting worse for a small customer group or a rare but expensive case. Promotion requires evidence that the dangerous slice stayed within its limit.

Treat labels as production assets. Two reviewers can disagree because the policy is vague, because the example lacks context, or because one reviewer is wrong. Record disagreements and resolve the policy question before using the case as truth. Otherwise the eval rewards whichever interpretation entered the file first.

Watch the eval set for contamination and drift. If developers see every locked answer, they will optimize to the test even without intending to. If live requests move into new languages, document types, or customer segments, the old set stops representing the approved population. Add new cases through a reviewed process, version the set, and keep prior versions so a changed score remains explainable.

Cost ceilings belong in the design

Set a maximum cost per accepted business outcome and a monthly exposure limit before usage grows. Model price per token is an input, not the cost of the service. The real bill includes retrieval, reranking, repeated calls, tool execution, retries, evaluation, storage, observability, and human review.

Teams routinely calculate cost from the happy path. The request uses one prompt, one response, and the cheapest listed rate. Live traffic adds long documents, duplicated context, timeouts, invalid structured output, fallback models, and users who press retry. A workflow that costs $0.08 in the demo can cross its business limit without any vendor price changing.

Measure cost per accepted outcome. If ten attempts cost $1.00 and only six pass validation, the relevant model cost is about $0.17 per accepted result before review and infrastructure. If a reviewer spends four minutes on each result, labor may dominate the model bill. This is why "AI is cheaper" cannot be an assumption in the investment case.

Put a cost envelope in configuration so the service can enforce it:

cost_policy:
  max_usd_per_request: 0.25
  max_usd_per_accepted_result: 0.40
  monthly_budget_usd: 12000
  alert_at_percent: 70
  degrade_at_percent: 90
  action_at_limit: "queue_for_review"

The exact values will differ, but the actions must be decided in advance. At the alert threshold, the owner investigates. At the degradation threshold, the service can shorten context, use a qualified cheaper model for low-risk cases, or reduce optional retries. At the hard limit, it queues work instead of silently spending past the budget.

Cost controls must preserve the eval threshold. Switching every request to a smaller model may fix the invoice by breaking the product. Test each degradation mode as its own configuration and publish its quality, latency, and cost. Finance needs a forecast, while operations needs predictable behavior when the forecast is wrong.

Track unit economics by segment. One tenant with unusually large files or one workflow with a high retry rate can subsidize itself from everyone else's average. Averages make that pattern disappear until the invoice forces an unpleasant customer conversation.

Model concurrency can create a second ceiling. A workflow may meet its per-task target and still overwhelm provider rate limits during a morning spike, leaving paid employees waiting and review queues growing. Test burst traffic, cap concurrent calls, and include queue time in the service objective. Buying a higher rate tier may be sensible, but it belongs in the forecast.

Reconcile estimates with invoices and internal labor after every release gate. Token counters can miss provider-specific billing details, and reviewers rarely record all interruption time. A weekly cost report should show requested work, accepted work, failed work, review minutes, and spend. That view makes waste visible before the monthly total becomes a surprise.

Reliability lives outside the model call

A production AI feature needs the same failure handling as any external dependency, plus controls for invalid and unsafe output. Model calls time out. Providers throttle. Structured responses fail schema checks. Retrieval returns nothing. Tools partly execute. The service must decide what happens in each case without asking an engineer to watch a console.

Start with an explicit state machine. received can move to processing, then to accepted, rejected, or review_required. A timeout can retry within a bounded policy; a schema failure can attempt one repair; a failed side effect must not repeat blindly. Store the state transition and request identifier so support can explain what happened.

Idempotency matters when the model can trigger actions. If a user retries an invoice workflow after a timeout, the system must not issue a second credit because the first response arrived late. Generate an idempotency token before the model call and pass it to the action boundary. The model may propose an action, but ordinary application code should validate and execute it.

Do not hide uncertainty with retries. Three samples and majority voting can improve some tasks, but they triple call cost and may produce three confident versions of the same error. Retries are appropriate for transport failures and explicitly tested recovery paths. They are a poor substitute for an abstention policy.

Observability should connect four records: the business request, model and prompt version, validation or eval result, and final business outcome. Log references rather than unrestricted prompt bodies when the content is sensitive. The purpose is to answer concrete questions: Which version produced this result? Which checks ran? Did a person override it? Did the promised action finish?

Write a rollback condition before release. A rise in a critical failure class, a cost breach, or a growing review queue can trigger rollback even while general availability looks healthy. Rolling back may mean disabling an agent action while preserving suggestions, returning to a previous prompt and model pair, or routing all cases to the existing manual process.

Human review is a queue, not a safety slogan

Human review works only when the team defines who reviews, what evidence they see, how quickly they respond, and what happens when the queue fills. "A human stays in the loop" often means the design has moved an unresolved risk onto an unnamed employee.

Calculate review capacity from arrival rate and handling time. If the system sends 300 cases per day to a queue and each takes four minutes, it has created twenty hours of daily review work. The proof may still look efficient because the engineer reviewed only five examples in the demo.

Reviewers need the source, proposed output, applicable rule, confidence or validation signals, and a clear set of actions. Do not show a mysterious score without its meaning. More information can also hurt: a long model explanation may anchor the reviewer on the model's reasoning instead of the source facts.

Sample accepted cases as well as rejected ones. Reviewing only low-confidence output measures whether the rejection gate works; it cannot find confident mistakes that passed. Use risk-based sampling, and feed confirmed failures back into the investigation eval set. Promote recurring failures into the locked set after the team agrees on expected behavior.

Define overflow behavior. The choices are business decisions:

1. Pause intake and tell users when processing will resume.
2. Fall back to the existing manual workflow.
3. Restrict the feature to lower-risk cases that can pass automatically.
4. Increase staffed review capacity under a preapproved plan.

Letting the queue age silently is not a fifth option. It converts a quality control into a service-level failure and makes cost per outcome look artificially low because the unfinished work has not been counted.

Review data also needs access and retention rules. Corrections may contain sensitive source text, employee notes, and policy decisions. Treat the review tool as part of the production data path, not as a temporary admin screen that escapes the controls applied elsewhere.

One owner must control the promotion decision

One business owner must have authority over scope, acceptance, budget, and shutdown. Committees can advise, and engineering can operate the service, but shared ownership usually means no one can make the uncomfortable decision to stop a popular demo.

The owner should come from the function that receives the outcome. A support leader should own a support assistant's operational result; a CTO should not own refund policy merely because the workflow uses a model. Engineering owns technical implementation and incident response. Security and legal own their approval domains. Finance confirms the budget. None of those roles replaces the outcome owner.

Give that person a one-page promotion packet with the hypothesis, eval results by failure class, data-flow approval, cost envelope, review capacity, rollback triggers, and unresolved exceptions. The decision is promote, hold, or stop. Avoid pilot forever, the informal state where real users depend on a service that still has experimental controls and temporary funding.

Responsibility must continue after launch. The owner reviews changes to models, prompts, tools, data sources, and user populations because each one can invalidate the original evidence. A provider upgrade is not routine maintenance when it changes behavior. Run the relevant regression set and record the promotion decision for the new configuration.

Vendors do not own your business result. A model provider can publish availability and safety documentation, but it cannot decide whether a wrong answer creates a tolerable support delay or an unacceptable financial action. Your team owns that translation from model behavior to business consequence.

Ownership without time and authority is decorative. If the named person cannot obtain data approval, reserve review staff, or stop a launch promised by sales, the organization has assigned a messenger rather than an owner. Put those rights in the promotion record and settle conflicts with other leaders before the service handles live work.

The owner also needs a recurring operating review. Monthly may suit a low-volume internal assistant; a financial action agent may need daily attention during rollout. Review failure classes, spend, queue age, overrides, incidents, and configuration changes. Cancel the meeting when the service ends, not when the launch excitement fades.

When I assess an AI initiative through oleg.is, I ask for the owner's name before I ask which model the team chose. The missing name predicts the demo-to-production gap more reliably than a weak prompt does.

Promotion should happen through controlled exposure

Move from proof to production by increasing exposure only when evidence supports the next boundary. A big launch mixes model risk, workflow risk, demand risk, and support risk in one event. A controlled release separates them and leaves room to stop.

A practical sequence has four gates. First, replay historical cases without affecting users and compare the proposed output with known outcomes. Second, run in shadow mode on live traffic while the existing workflow remains authoritative. Third, expose the feature to a small, eligible group with review and an immediate fallback. Fourth, expand by risk segment while watching the predeclared thresholds.

Each gate needs an entry condition and an exit decision. Time alone is not evidence. "We ran shadow mode for two weeks" says less than "we processed the required sample across every defined segment, with zero critical privacy failures and review demand below staffed capacity." Use the measures the owner approved, not a calendar milestone.

Keep changes small during promotion. If the team changes the model, prompt, retrieval index, and user interface together, a failure teaches almost nothing. Version the full configuration and promote one tested bundle. If an urgent provider change forces a new model, rerun the relevant evals before expanding exposure.

The first production version should often do less than the demo. Remove risky actions, narrow the input population, cap document size, or require review for a sensitive segment. Scope is a control, not an embarrassment. A narrow service with a measured result can expand; a broad service with unexplained failures loses trust quickly.

Rehearse failure before the first exposed group arrives. Force a provider timeout, reject an invalid response, exhaust the cost budget, and simulate an unavailable review team. Confirm that alerts reach the assigned person and that the fallback completes rather than merely appearing in a document. A rollback plan that nobody has executed is still a hypothesis.

Customer communication belongs in the gate as well. Users need to know when output is generated, what they should verify, where to report an error, and whether a human will review the case. Do not bury operational limits in a policy page. Put the instruction at the decision point, using language that matches the actual control.

Do not declare success at launch. The proof closes only after the team observes real outcomes, reconciles spend, processes review feedback, tests rollback, and confirms that an owner can run the service without the original prototype engineer. If that engineer still carries the system through private knowledge and manual fixes, production has not begun.

Kill weak proofs before they become expensive pilots

Stop a proof when it misses a business threshold, depends on unavailable data rights, requires unaffordable review, or lacks an owner willing to accept the remaining risk. Killing it is a sound portfolio decision. Extending it because the demo impressed an executive spends more money without changing the constraint.

The popular advice to "just get an AI demo in users' hands" is incomplete. Speed helps when the team has named the decision the demo will inform. Without acceptance and stop conditions, user exposure creates anecdotes, pressure, and accidental dependency instead of evidence.

Use a short pre-demo review. The owner should be able to answer five questions:

• Which exact population may use this, and which cases must it reject?
• Which failures block promotion even if the average score improves?
• What is the maximum cost per accepted outcome and per month?
• Who handles review and incidents when volume exceeds the forecast?
• Which single person can pause the service?

If the room cannot answer them, cancel the polished presentation and use the time to settle the operating decisions. That is not bureaucracy. It is cheaper than discovering after integration that the only safe workflow needs more reviewers than the current team employs.

For founders who need an outside decision process, a Team & AI Audit can examine the workflow, team, costs, and production controls in five business days for a fixed $5,000. The useful output is not another demo. It is a defensible choice about what to promote, redesign, or stop.

An AI proof earns production traffic when the company can explain its data path, reproduce its eval result, enforce its spend, absorb its exceptions, and identify the person who can say no. Until then, the demo is evidence of possibility, and possibility is only the beginning.