Product principles for small AI teams: what to build

Why teams of two pick the wrong work

Teams of two often choose the wrong work because a good demo feels much closer to a real product than it is. You can make a model do something impressive in 20 minutes. Making that same behavior reliable for real users can take weeks.

That gap fools smart teams all the time. A prototype answers one clean prompt on a good day. A shipped feature has to handle vague inputs, bad formatting, rate limits, provider errors, and users who expect roughly the same result tomorrow.

AI adds another trap. When you have several models, you do not get more clarity. You get more tempting options. One model writes better. Another is cheaper. A third works well on one narrow task. Every option looks promising, so the team keeps comparing, tuning, and switching instead of deciding what belongs in the product.

Messy product work often starts with something that looked obvious in a demo. A founder sees a strong result and says, "Let's add this." Then the hidden work shows up: prompt versioning, fallback rules, logs, human review, edge cases, and support questions when outputs disagree.

A quick experiment and a real feature are different kinds of work. An experiment only has to be interesting once. A feature has to survive months of use. Someone has to maintain it, explain it, test it after model updates, and decide what happens when it fails.

Small teams feel this faster than larger ones because they cannot bury bad bets in a long roadmap. If two people spend a month on the wrong feature, that month is gone.

A small AI team needs a simple filter before anyone writes production code. It does not need theory. It needs three practical questions: will this stay consistent enough to trust, can we inspect what happened later, and can we undo it without pain? If the answer is weak, keep it as an experiment.

The three rules that sort ideas fast

When two people own product, prompts, and production, time disappears fast. The easiest way to stop circular debate is to judge every idea with three checks: does it stay steady, can you inspect it, and can you undo it?

Consistency comes first because unstable output creates fake progress. A feature can look great in one demo and fail on the next five tries. If similar inputs produce different answers, different actions, or different formatting for no clear reason, treat that as a warning.

Auditability comes next. Someone on the team has to see what happened and why. That means you can inspect the model used, the prompt version, the tools it called, the data it touched, and the output it returned. If a result looks wrong and nobody can trace the path behind it, fixing it turns into guessing.

Reversibility keeps small mistakes small. Some changes are easy to roll back, like swapping a prompt, hiding a button, or turning off one automation. Others leave a mess behind, like changing a workflow customers depend on or writing bad data into a live system. The harder a change is to undo, the more proof you need before you ship it.

A team using Claude, GPT, and an open-source model can apply these rules in a few minutes. Run the same task on a small set of similar inputs and see whether the outputs stay within a clear range. Save enough detail so either person can retrace the result later. Then ask a simple question: if this fails on Friday night, can one person remove it fast?

These rules cut debate because they replace opinion with evidence. Instead of arguing about whether an idea feels smart, the team asks three direct questions. If outputs drift, logs are thin, or rollback looks painful, the idea does not get production time yet.

That may sound strict, but it usually saves time. Teams that work with AI-heavy products learn this early: the flashy idea is rarely the one worth shipping first. The better bet is the change you can repeat, inspect, and reverse without drama.

What consistency looks like in practice

Consistency means the tool gives roughly the same level of quality every time a user runs it. A flashy demo does not matter much if the next five runs fall apart.

For a team of two, repeatable output usually beats the one perfect result that took twelve prompt edits and a lucky model run. If one model writes a clean customer reply once but misses tone, facts, or structure on the next three attempts, you do not have a product yet. You have a demo.

A simple way to test this is to run the same task several times with small changes. Change the prompt wording a little. Swap one data source. Try a second model. Use messy input instead of neat input. If quality swings hard, users will notice before you do.

Clean examples are misleading. Real users paste broken text, vague requests, half-finished notes, and data with gaps. That is why edge cases matter more than polished test cases. If your tool handles only the easy version, it will feel random in real use.

A small test set is enough. Use five normal cases that reflect everyday work, three messy cases with missing or noisy input, two failure cases that should trigger a safe fallback, and one case you already know each model handles differently.

Drift often shows up in quiet ways. Claude may stay structured while GPT gets more creative. An open-source model may do fine until the retrieved context gets long. A prompt tweak that improves speed may also reduce accuracy. Even a small data change, like a different retrieval chunk or document version, can shift an answer enough to break trust.

Users accept some variation. They do not expect every sentence to sound the same. They do expect the tool to stay within a clear range. If your app writes support drafts, users may accept different wording but not different facts. If it tags tickets, they may accept 5 percent disagreement, not 25 percent.

That is the practical rule: define the range of acceptable variation before you spend production time. If you cannot describe that range in plain language, the feature is still too loose.

How to make AI work auditable

If a model can affect customers, money, code, or internal records, you need a trail that a human can read in two minutes. Auditability is not a big-company habit. It is how a two-person team avoids long arguments after something goes wrong.

Start by recording the same facts every time the model does real work: what the user or system asked for, which model and version ran the task, the exact prompt or template used, the raw output, and the action taken after that output. That record should sit next to the task, not buried in logs that only an engineer can search.

If a founder asks, "Why did the assistant send this reply?" the answer should be visible without opening three tools and reading a wall of JSON.

Review notes matter too, but they should stay short and plain. A non-technical founder should understand what happened, why the team accepted or rejected the result, and what changed after review. A good note sounds like this: "The model used old pricing text and drafted the wrong refund answer. We updated the source note and added a check before sending."

Ownership also has to be clear. When a bad result hurts a customer, one person should own the fix, the follow-up, and the rule change. If ownership is fuzzy, the same bug comes back a week later. Small teams do better with names than roles. "Sam reviews support prompts" is clearer than "support owns it."

Fast review beats perfect review. Set a short rhythm, maybe 20 minutes twice a week, and bring real failures. Look at the bad output, the prompt, the source data, and the action that followed. Then decide one change: tighten the prompt, block a risky action, add a human check, or remove the task from production.

If your team cannot replay a bad result and explain it in plain English, the workflow is not ready for production.

A step-by-step test for new ideas

A team of two does not need a long planning ritual. It needs a short test that kills weak ideas early and gives the strong ones a fair shot.

This matters even more when the team uses several models in one product. Claude may do one task well, GPT may handle another, and an open-source model may be cheaper for routine work. If the test is loose, the product gets messy fast.

Start with one sentence. Write the user task in plain language, with a clear actor, action, and result. "A support agent pastes a ticket and gets a draft reply in 30 seconds" is clear. "Improve support with AI" is too vague to test.

Then try that task on real cases, not invented ones. Use a small set that includes an easy case, a messy case, and one that usually causes trouble. Run the same task through each model or prompt setup you are considering. If the output changes too much from one similar case to the next, keep it out of production.

Save the evidence in one place. A document, spreadsheet, or issue is enough. Record the input, the exact prompt or tool settings, the output, and a short reviewer note. When something fails later, you will know what changed and why the team approved it.

Before you build more around it, ask one rollback question: what would you need to undo after a bad week? If the answer includes retraining staff, cleaning broken records, or untangling product logic, the change is harder to reverse than it looks. Keep it smaller until rollback is cheap.

Last, make both teammates explain the choice in plain language. Each person should be able to say what the task is, why this version passed, and where it may fail. If the explanation turns fuzzy, the idea is not ready.

This test is simple, and that is the point. Ten careful examples and a page of notes can save a small team from a month of cleanup.

A simple example with two people and three models

Imagine two people running support for a small SaaS company. They want one tool that reads a ticket, checks a short policy file, and drafts a reply for the agent. They do not start with full automation. The first version only writes a draft that a human can edit, approve, or delete.

That choice avoids a common mistake. A bad draft is cheap to fix. A bad sent message can trigger a refund fight, a public complaint, or three more emails.

They test three models on the same set of 50 real tickets. The fast model replies in a couple of seconds, but it skips details when a thread gets messy. The cheaper model does fine on basic shipping and account questions, yet it drifts when the policy text gets long. The stronger model reads long threads better and follows refund rules more closely, but it costs more and takes longer.

So they split the work. The fast model tags the ticket type. The cheaper model drafts replies for plain, low-risk cases. The stronger model handles refunds, billing disputes, and any ticket where a wrong answer would create cleanup work.

They keep every test run in one place so they can review it later. Their record includes the ticket ID, the model used, the prompt version, the policy snippets pulled into the prompt, the draft reply, and the human edits and final send decision.

The app stores those records in a simple database table. The team keeps review notes in a GitLab issue for each test batch, with short labels like "wrong policy," "too vague," and "good but too long." If a pattern keeps showing up, they change the prompt or routing rule before they touch anything else.

They ship only the draft step because they can reverse it in seconds. If the model picks the wrong tone, the agent rewrites it. If the routing rule fails, they switch the ticket back to manual handling. They do not ship auto-send, auto-refund, or policy changes yet, because those actions are harder to unwind.

That is a sensible decision framework for a team this small: ship the part you can inspect, log, and roll back before lunch.

Mistakes that create long-term pain

The worst mistakes often start as small wins. A feature gives one brilliant answer in a demo, everyone gets excited, and it jumps straight into the product. That is how teams end up building around a moment instead of a pattern.

One good output proves almost nothing. Models can look smart for ten minutes and still fail on the boring cases users hit every day. If two people spend a week building around a flashy demo, they often learn too late that the result is too unstable to trust.

Another common mistake is changing too many things at once. If you rewrite the prompt, swap the model, and adjust the UI in the same release, you lose the trail. When results improve, you do not know why. When they get worse, you do not know what to undo.

That gets expensive quickly. Support issues take longer to trace. Testing turns into guesswork. User feedback gets muddy because several changes shipped together. Rollbacks become risky because nobody knows which part caused the problem.

Logs often feel optional when the first users seem happy. That feeling does not last. Once the team gets a strange output, a missing answer, or a cost spike, memory is useless. You need the prompt version, model name, input, output, and the action taken after it. Without that record, every bug becomes a debate.

Silent model decisions cause a different kind of damage. If the model can approve, reject, classify, or send something without a human check, small errors spread quietly. A wrong summary is annoying. A wrong customer message, bad routing choice, or false risk flag can create days of cleanup.

Rollback plans get pushed aside for the same reason logs do: early progress feels smooth. But reversibility is not extra work. It is part of the feature. If you cannot switch back to the last prompt, last model, or last rule set in minutes, you are gambling with production.

A two-person team usually does better with one simple habit: change one thing, log everything, and keep a clean way back. It sounds cautious. Over a month, it is much faster than cleaning up avoidable mess.

A quick checklist before you spend production time

A feature does not earn production time because a demo looked smart once. It earns it when your team can trust it on an ordinary Tuesday, with rushed inputs, unclear prompts, and users who notice every odd result.

Good product rules are boring on purpose. Before you wire anything into the product, run five checks.

First, try ten similar inputs. If the answers swing too much, keep the feature out of the product or narrow the task until the outputs settle.

Second, check whether you can explain the result later. You need the prompt, model version, tools used, retrieved context, and final output in one place.

Third, test the rollback path. If the model fails, can you switch to a simpler rule, an older prompt, or human review without breaking the flow?

Fourth, judge trust, not just accuracy. Users forgive a plain answer more easily than a clever answer that changes tone, format, or facts each time.

Fifth, use boring examples. Typos, half-filled forms, mixed languages, repeated fields, and vague requests expose weak ideas fast.

Small teams usually get in trouble when they test only clean cases. Real usage is messy. Someone pastes a customer note with missing dates, or asks the same question three slightly different ways, and now the product behaves like three different tools.

Audit trails matter even for tiny products. If a user asks, "Why did it do that?" your team should answer in minutes, not after a day of guesswork. Log the input, prompt, model, context, and any post-processing rules.

Reversibility is the part teams skip most often. They glue a model into signup, billing, support, or search, then learn too late that undoing it means rewiring everything. Keep the model behind a clear boundary so you can turn it off and still keep the product usable.

If a feature passes all five checks, it deserves a week of real build time. If it fails two, send it back to the sandbox. That choice may feel strict, but it is cheaper than cleaning up a clever mistake in public.

Next steps for a team of two

A team of two does not need a bigger roadmap. It needs a stricter filter. Pick one idea you are discussing right now and score it against consistency, auditability, and reversibility. Use a simple scale like pass, unclear, or fail. If you cannot score it in five minutes, the idea is still too fuzzy for production work.

Be harsh with demos. A feature can look smart in a live test and still be a bad bet. If it fails two of the three rules, stop it. That usually means users will get uneven results, your team will not know why the model behaved a certain way, or you will get stuck with a change that is painful to unwind.

Most small teams need only a short release habit. Write the idea in one sentence. Check whether the output stays consistent for the same kind of input. Check whether a person can review what happened later. Check whether you can roll it back in one day without breaking other work. Record the decision in a shared note.

That routine should happen before every release, even small ones. Ten calm minutes before shipping can save a week of cleanup. Over time, the team gets faster because fewer weak ideas make it into the build.

One shared note is often enough. Add the feature name, the models involved, what the team expects, how to inspect the output, and how to turn it off. It is boring work, but boring work keeps small teams out of trouble.

If you are building with several models, this matters even more. One model may write well, another may classify better, and a third may be cheaper. The mix can work, but only if each step stays predictable and easy to inspect.

Some teams need an outside view to set this up without adding process for its own sake. Oleg Sotnikov at oleg.is works with startups and small businesses on AI product decisions, lean technical processes, and practical ways to ship new systems without making rollback and review harder than they need to be.