# Data readiness for AI before the first build

> Assess data readiness for AI through concrete checks for quality, access, ownership, governance, labels, drift, and production controls.

Most AI projects that I see fail early do not have a model problem. They have a decision problem hidden inside a data problem. The team can train a convincing prototype, but nobody can say which records it may use, what a correct answer looks like, who owns a bad outcome, or whether production can reproduce the demo.

Data readiness for AI means that a named owner can show, with evidence, that the available data is fit for one defined decision under real operating conditions. It does not mean that the company has cleaned every table or bought a data catalog. Readiness belongs to a use case, and it expires when the process, population, or source system changes.

I use a gate review before anyone chooses a model or signs a platform contract. The review tests the decision, data quality, access path, legal purpose, labels, lineage, production feedback, and stop conditions. A weak result may still justify a small discovery experiment. It does not justify a production promise.

## Readiness belongs to a decision, not a dataset

A dataset cannot be "ready for AI" in the abstract. It can be suitable for predicting late invoices next week and unsuitable for deciding which customers receive a credit limit. The same columns support both tasks, but the cost of an error, the permitted purpose, the needed freshness, and the people affected are different.

Teams often blur three separate states. Available data exists somewhere. Usable data can be retrieved and interpreted by the project team. Data ready for production can be retrieved on schedule, under approved permissions, with monitored quality and a response when it fails. A CSV exported by an analyst proves availability. It proves neither repeatable access nor production readiness.

Start with a decision contract in one sentence:

> For each [subject], at [decision time], the system will produce [output] for [operator or downstream system], which will take [allowed action], while a named owner remains accountable for [harm or exception].

"Score leads" is too vague. "At 08:00 each workday, rank new inbound leads for the sales operations queue; a salesperson chooses whom to contact, and the sales director owns complaints and missed priority accounts" is testable. It defines the unit, timing, consumer, action, and owner. It also exposes whether the result is advice or an automated decision. That distinction changes the required controls.

The decision contract prevents an expensive habit: collecting everything first and deciding what to build later. That habit is popular because storage feels like progress and postpones disagreements. It is wrong because quality has no meaning without a field, population, time window, and tolerated error. Keep source data that policy permits, but assess only the slice a real decision needs.

For each use case, write down four failure costs: a false positive, a false negative, no answer, and a late answer. "No answer" deserves its own line. Many teams tune model accuracy while the pipeline silently drops records, which converts uncertain predictions into invisible exclusions.

## Define the evidence before inspecting columns

The readiness review should begin with acceptance evidence, because a team that starts by browsing tables will adapt the use case to whatever data looks convenient. That produces a polished demo and a weak product.

Create a compact assessment record with these fields:

1. Decision, subject, and decision time.
2. Business owner and technical owner.
3. Allowed action, forbidden action, and human override.
4. Success measure plus an operational baseline.
5. Maximum acceptable harm, delay, and abstention rate.

The operational baseline matters more than a benchmark copied from a paper. If people currently resolve 70 percent of support tickets within a day, compare the proposed system with that process, including tickets it cannot classify. Do not compare a clean test set with an imagined perfect employee. Use your own observed baseline and document how you measured it.

Next, specify the evaluation set before model development. Define its time range, included populations, exclusions, label source, and minimum sample for each important subgroup. Do not let the training query become the evaluation query with a different random seed. Random splits leak time and process artifacts whenever future records resemble past records from the same customer, machine, or case.

A holdout split by time is often the honest default for operational AI. Train on data available before a cutoff and test on later decisions. If one subject can produce many rows, group all rows for that subject on one side of the split. Otherwise, the model may recognize a customer rather than learn a pattern that transfers to a new customer.

Acceptance evidence also needs a shadow result. During a shadow period, the system produces outputs but does not drive the action. Compare those outputs with the existing process, record disagreements, and ask an owner to classify the expensive ones. An aggregate score cannot explain whether errors cluster around new customers, sparse histories, one acquisition channel, or a workflow change.

Build a source map beside the evaluation plan. For each field, name the system of record, table or event, business definition, update trigger, earliest reliable date, join path, and owner who can resolve a discrepancy. Mark fields copied from another system, because the convenient warehouse column may lag or transform the authoritative event. If two sources claim authority, choose one for this decision and record how reconciliation works.

Then trace a small sample by hand. Pick a normal case, a recent case, a case with missing fields, a corrected case, and a case near the decision boundary. Follow each from the original event through transformations to the proposed feature and label. This exercise routinely finds timezone shifts, reused identifiers, cancelled events that remain active, and updates that overwrite history. A profile over the final table rarely exposes those errors.

Document exclusions with counts and reasons. If the project removes records because their labels are missing, identifiers do not join, text uses an unsupported language, or a source arrived late, report each removed population separately. The evaluation cannot claim coverage of people whom preprocessing discarded. An exclusion can be correct, but the owner must know who receives no answer and what operational path handles them.

Finally, compare the requested fields with the decision contract. Remove fields that do not support an agreed feature, eligibility rule, audit need, or evaluation slice. "We may need it later" is not a purpose. A smaller field set reduces access work and makes future changes easier to review, while an undocumented grab bag lets the use case expand without a new decision.

If the owner cannot set an acceptable error or name a safe fallback, stop. More data will not settle a policy decision. The project needs product and risk ownership before it needs machine learning.

## Data quality needs thresholds and failure actions

Data quality for AI is a set of observable tests tied to the decision contract. Words such as "clean," "complete," and "trusted" hide the threshold and the response. A production check must say what it measures, where it runs, when it runs, and what happens after failure.

Use six dimensions, but define them in local terms. Completeness asks whether required values exist. Validity asks whether values obey domain rules. Uniqueness asks whether the decision unit appears once where it should. Consistency asks whether sources agree. Timeliness asks whether data arrives before the decision. Representativeness asks whether the evaluation population covers the production population that will receive outcomes.

The last dimension is where standard warehouse checks stop too early. A table can pass null, range, and duplicate tests while excluding customers who joined through a new channel. The SQL is valid; the decision is biased toward yesterday's intake process.

Here is a small check pack for a daily lead-ranking job. Replace the thresholds with values agreed by the owner, not numbers chosen after a failed run.

```sql
/* One row with counts and rates, stored as a quality run artifact */
SELECT
  CURRENT_DATE AS run_date,
  COUNT(*) AS rows_seen,
  SUM(CASE WHEN lead_id IS NULL THEN 1 ELSE 0 END) AS missing_ids,
  COUNT(*) - COUNT(DISTINCT lead_id) AS duplicate_ids,
  AVG(CASE WHEN created_at >= CURRENT_TIMESTAMP - INTERVAL '48 hours'
           THEN 1.0 ELSE 0.0 END) AS fresh_rate,
  AVG(CASE WHEN acquisition_channel IN
      ('organic', 'referral', 'paid', 'partner')
      THEN 1.0 ELSE 0.0 END) AS known_channel_rate
FROM candidate_leads;
```

The result should have an ordinary tabular shape that a run log can retain:

```text
run_date  | rows_seen | missing_ids | duplicate_ids | fresh_rate | known_channel_rate
2026-08-09| 1842      | 0           | 3             | 0.992      | 0.978
```

That output is not a pass by itself. A policy might block publishing when `missing_ids > 0`, quarantine duplicate IDs, warn when freshness falls below an agreed level, and compare channel shares with the approved reference window. Store the query version, source snapshot or partition identifiers, result, threshold version, and disposition. Otherwise, a green dashboard cannot tell an incident reviewer what actually passed.

Profile distributions as well as schema. Record category shares, numerical quantiles, zero rates, text lengths, and the age of the newest event. Compare them by time and by populations that matter to the decision. Choose those populations from the harm analysis, not merely from columns that are easy to group.

Missing values need causes. "Unknown because the customer skipped the form" differs from "missing because an ingestion job failed" and from "not applicable for this product tier." Collapsing all three into null teaches the model a pattern that can change when the form or pipeline changes. Preserve a reason code when the process knows it.

## Access must work as the production identity

Data access is ready only when the production workload can retrieve the approved fields, at the required time, through a supported path. An engineer querying a replica with personal credentials is not an access test. It is a preview.

Run an access rehearsal with the actual service identity in a test environment that mirrors permissions. The rehearsal should cover five actions:

1. Read the documented sources and only the approved columns.
2. Join them using stable identifiers rather than names or mutable email addresses.
3. Retrieve enough history to build features without bypassing retention rules.
4. Write outputs to the intended destination with an idempotency identifier.
5. Emit audit records that connect the input snapshot, code version, model version, and output batch.

Time the complete retrieval, including queues and upstream delays. A query that runs in four minutes against a prepared snapshot may miss a decision window of five minutes when the upstream feed arrives three minutes late. Measure freshness at the source event, ingestion boundary, transformed table, and feature read. "Pipeline completed" says nothing about how old its input was.

Use least privilege by creating a role for this purpose. Avoid copying broad analyst access into a service account because it is faster. That shortcut quietly expands the project scope: a later feature request can use sensitive columns without a new review because the credential already exposes them.

The team also needs a development path that does not normalize raw production downloads. Prefer masked or synthetic data for routine engineering, then use controlled jobs for approved evaluation. If developers must inspect raw records to debug an exception, limit the access by time, log it, and tie it to a case. A policy nobody can follow during an incident will be bypassed during the first incident.

Test revocation and source failure. Disable the role and confirm the job fails closed without using cached credentials. Remove one upstream partition and confirm the system abstains, delays, or falls back as the decision contract requires. If the pipeline quietly scores an incomplete population, your availability feature has become a data quality bug.

## Governance must name purpose, authority, and deletion

Governance is ready when a named owner can explain why each sensitive field is used, who may use it, how long derived data persists, and how a person can challenge an outcome where policy or law requires it. A slide that says "compliant" answers none of those questions.

Build a purpose table for every field. For every input and derived feature, record its business purpose, source authority, sensitivity class, approved users, retention period, deletion behavior, and whether it influences an outcome about a person. Derived features belong in the table too. A risk score can reveal more about a person than any single source column, so treating it as harmless output is a serious category error.

NIST's AI Risk Management Framework separates the work into Govern, Map, Measure, and Manage. That sequence is useful because measurement does not replace governance. A team can calculate excellent error metrics for a use that the business never authorized. I would add one practical demand: attach every governance claim to an owner and an inspectable artifact. "Legal reviewed it" should point to the approved purpose, conditions, date, and reviewer, not an old meeting invite.

For personal data, purpose limitation and data minimization are operational constraints, not wording for a privacy notice. Ask whether the decision can work without each field, whether a less sensitive proxy would create its own unfairness, and whether training creates a new purpose beyond the transaction that collected the data. Get qualified legal advice for the jurisdictions and decisions involved. An engineering checklist cannot decide legal authority.

Deletion must cross boundaries. Deleting a source row may leave feature tables, training snapshots, label exports, embeddings, caches, model inputs in logs, and analyst files. Map which copies can be deleted directly, which age out under a documented schedule, and which require model retraining or another approved response. Do not promise machine unlearning unless the implemented system can perform and verify it.

Record the review as a decision log with four possible outcomes: approved, approved with conditions, discovery only, or rejected. Conditions need deadlines and owners. "Approved pending governance" usually means nobody will revisit governance after the demo earns attention.

## Labels are claims about reality

A label is not ground truth merely because it sits in a column named `outcome`. It is a claim produced by a process, and that process has incentives, delays, disagreements, and missing cases.

Trace each label to the event or judgment that created it. For a churn model, "cancelled within 30 days" may be reproducible, while "likely to leave" may reflect an account manager's intuition. For support routing, the queue where a ticket closed may show staffing practice rather than the category that should have handled it. Training on that label reproduces the old routing constraints.

Audit four properties: definition stability, observation delay, coverage, and disagreement. Definition stability asks whether policy or software changed the meaning over time. Observation delay asks when the outcome becomes knowable. Coverage asks which subjects never receive a label. Disagreement asks whether two qualified reviewers reach the same answer on ambiguous cases.

Sample records from different periods and populations, then have reviewers label them without seeing the historical outcome. Do not use agreement as a ceremonial metric. Open disagreements and classify their cause: unclear instructions, missing context, legitimate ambiguity, or reviewer error. Rewrite the guide and repeat the sample until the remaining ambiguity matches the product's planned abstention or review path.

Watch for selective labels. A fraud investigation outcome exists only for transactions the old system sent to investigators. A sales outcome may exist only for leads a salesperson contacted. The unlabeled remainder is not a random sample. Treating it as negative bakes the previous policy into the new model and makes evaluation look better than deployment will be.

Label leakage can be embarrassingly subtle. A field such as `case_closed_reason` may perfectly predict escalation because staff fills it after escalation. Freeze the feature view at the decision time from the contract, and reject any value that becomes available later. Checking column names is insufficient; you need event timestamps and process knowledge.

When reliable labels do not exist, say so. The right first product may be retrieval, clustering for analyst exploration, or an assistant that drafts without making the decision. Renaming weak supervision as "ground truth" does not make a classifier safe.

## A data contract must survive production change

Readiness expires unless the team can detect changes in sources, meaning, and population before those changes alter decisions. A schema registry catches type changes. It will not catch a CRM administrator who redefines "qualified," a new region that uses different defaults, or a timestamp that now records processing time instead of event time.

Write a compact data contract that includes semantic and operational promises. This fragment shows the level of specificity I expect:

```yaml
dataset: candidate_leads_v3
owner: revenue_operations
decision_time: "08:00 America/Los_Angeles on business days"
entity_key: lead_id
event_time: created_at
freshness:
  newest_event_max_age_minutes: 90
required_fields:
  lead_id: {null_rate_max: 0}
  acquisition_channel: {allowed_set_ref: channel_policy_v4}
population:
  included_regions: [US, CA]
on_breach:
  action: stop_publish
  notify: [revenue_operations, ai_on_call]
```

The contract prevents several failures at once. It distinguishes event time from run time, assigns a business owner, versions a changing category policy, limits the approved population, and states that a breach stops publication. A schema with field types would miss most of that.

Version the transformation code, feature definitions, training data reference, label definition, evaluation set, model artifact, and decision policy together in a release record. You do not need one vendor to hold all of it. You need identifiers that let an incident reviewer reconstruct which inputs and rules produced a disputed output.

Monitor input drift, but do not let drift alerts make decisions by themselves. A changed distribution may be harmless seasonality, a successful marketing campaign, a sensor failure, or a new affected population. The owner must connect the alert to outcome quality and the decision contract. Set a response: inspect, shadow, restrict a population, roll back, or stop.

Feedback data needs the same skepticism as labels. If users accept an AI draft because editing is tedious, acceptance does not prove correctness. If salespeople call the highest-ranked leads first, later conversion data reflects the ranking policy. Log exposure, human action, override, delayed outcome, and missing outcome separately so the next evaluation can account for the loop.

## Score readiness without averaging away a blocker

A readiness score should organize evidence, not turn an unsafe project into a 74 percent green badge. Use gates first, then a score for prioritization.

I use eight categories: decision, quality, access, governance, labels, evaluation, production controls, and ownership. Score each from 0 to 3:

- Score 0: unknown or contradicted by evidence.
- Score 1: documented assumption, not tested.
- Score 2: tested once on a representative slice.
- Score 3: repeated, monitored, owned, and tied to a failure response.

Keep the evidence link or artifact identifier beside every score. A two without a query result, approval record, rehearsal log, or signed decision is a one.

Do not average critical zeros. The project cannot enter production if it has no accountable business owner, no lawful or approved purpose, no evaluation that matches decision time, no fallback for missing data, or no way to stop outputs. These are gates. A high quality score cannot compensate for absent authority, just as a signed approval cannot compensate for labels created after the decision.

Use the total only after gates pass. A low score can fund focused discovery. For example, weak label evidence may justify a relabeling sample over two weeks, while weak access may justify a rehearsal with the production identity. Tie discovery spending to the uncertainty it removes and the decision that follows. Do not fund a prototype with no deadline whose success criterion is that stakeholders like the demo.

The assessment should end with one of four calls:

- Proceed to a bounded experiment with named acceptance tests.
- Proceed only after listed conditions pass.
- Change the use case to reduce data or decision risk.
- Stop because the evidence cannot support the intended action.

For founders who want an outside challenge, the Team & AI Audit at oleg.is examines the team and AI opportunity over five business days for a fixed $5,000 fee, with at least $50,000 per year in identified savings or the audit is free. That offer does not make an unready use case ready; it gives the readiness gaps, engineering economics, and ownership questions a deadline and an accountable review.

## The first review should try to kill the project

The most useful first review tries to disprove readiness before a prototype creates political momentum. Give a small group the decision contract, source inventory, proposed label, evaluation design, purpose table, access rehearsal, and failure policy. Ask them to find one population the data misses, one feature available after decision time, one permission that is too broad, and one failure that the fallback cannot handle.

Run the review with the business owner, data owner, security or privacy representative, and the engineer who will carry the pager. The pager owner often asks the best question: "What will I see at 02:00 when this feed is wrong?" Answer with a monitor, an artifact, and an action, not with confidence in the model team.

The review can finish in a working session if the evidence already exists. More often it produces a short evidence backlog. Order that backlog by fatal uncertainty. Legal purpose, label validity, population coverage, and production access can end the project, so test them before tuning features or comparing model vendors.

Refuse three common substitutions. A successful notebook is not an access rehearsal. A data catalog entry is not proof of meaning or permission. An aggregate accuracy number is not an evaluation of operational harm. Each substitution removes the exact context that production adds.

Set an expiry date on the decision. Reopen the assessment when the use case changes, a new population enters, a source changes meaning, permissions expand, outcome quality moves beyond its limit, or the fallback fails. Even a mature system can become unready in one release.

The hard call is sometimes to stop. If nobody owns the harm, the labels encode an old policy, or the system cannot know when its input is incomplete, another model will only make the demo more persuasive. Kill that version of the project, repair the evidence, and bring back a narrower decision that production can defend.