AI output quality drops when prompts hide product decisions

Why "use common sense" causes bad output

When a prompt says "use common sense," the model does not look up your product rules. It fills the gap with averages. Those averages come from broad patterns in its training, not from the way your team wants the product to behave.

That sounds harmless until the missing choice actually matters. A support bot may answer in a friendly tone one time and a strict tone the next. A content tool may shorten text for one user but add extra detail for another. The prompt looks the same, but the unstated rule changes the result.

This is one reason AI output quality drops even when the prompt seems clear to the team that wrote it. People carry silent assumptions. They know what counts as "too long," "too informal," or "safe enough." The model does not. If you do not spell those choices out, it guesses.

Two requests can be almost identical and still get different answers because the model has to pick between several reasonable paths. Should it ask a follow-up question or act right away? Should it refuse a risky request or offer a limited version? Should it sound like a coach, a support agent, or a neutral editor? Humans settle these choices with product judgment. The model can only infer.

Hidden decisions usually show up in four places:

• tone and level of formality
• limits on length, scope, and what to include
• edge cases, exceptions, and fallback behavior
• refusal rules and safety boundaries

If those rules stay vague, teams often try to fix the problem later with more examples or model tuning. That can backfire. Tuning does not clarify a fuzzy decision. It can freeze that fuzziness into the system, so the model repeats the same wrong assumption more consistently.

A simple product case makes this obvious. Imagine a customer support assistant for a SaaS tool. The prompt says, "Use common sense and help the user solve the problem quickly." One user asks for a refund after a failed setup. Does the assistant apologize, explain policy, offer troubleshooting, or approve the refund path? Each option can seem sensible. The right answer depends on business rules, account status, refund windows, and how much authority the assistant has.

Once you replace "common sense" with direct rules, the output gets steadier. The model stops guessing what your product team meant. It starts following choices your team actually made.

What teams often leave unstated

Teams often think they gave the model enough direction when they explained the task. They usually skipped the product choices that shape the answer. That gap is where AI output quality drops.

A prompt may say "answer customer questions" and still miss the part that matters most: what the user is trying to do right now. Does the answer need to help them buy, fix a problem, compare options, or stay safe? If two goals conflict, the model needs an order. A support bot should solve the issue before it upsells. A sales assistant should qualify the lead before it explains every feature.

Refusal and escalation rules are another blind spot. Many teams write polite tone rules, but they do not say when the model must stop and hand the case to a person. That creates bad guesses. If a user asks for a contract term, a refund promise, medical advice, or a production change on a live system, the model should not improvise. It should refuse, ask for the missing authority, or route the case to a human.

Missing facts need rules too. Dates, prices, names, and quantities sound small, but they change the whole reply. When a user asks, "Can you start next week?" the model should know whether to ask for a date, use the user's time zone, or avoid making any commitment. If pricing depends on team size or scope, the model should say that plainly instead of inventing a number. If a name is missing, it can use a neutral greeting rather than guessing.

The words inside the product matter more than teams expect. If your business says "Fractional CTO", "technical due diligence", or "AI-augmented development", the model should use those terms and avoid made-up labels. Consistent language keeps answers clear and makes the product feel like one product, not five different voices stitched together.

Exceptions need the same level of detail. Most teams say "be flexible" and stop there. That is too vague. Spell out the edge cases:

• urgent security or outage reports go to a human fast
• legal or billing commitments need approved wording
• missing project details trigger a short follow-up question
• returning customers can skip questions you already know

A simple rule set like this beats "use common sense in prompts" every time. Oleg's work with startups and Fractional CTO clients shows why: once the team writes down these product decisions, the model stops guessing and starts acting like part of the business.

How to spot vague instructions in a prompt

A prompt can look detailed and still hurt AI output quality if it hides decisions behind soft language. The fastest way to find the problem is to read it like a new hire on their first day. If a sentence needs background knowledge, team history, or a manager's taste to make sense, it is vague.

Start by searching for phrases that sound harmless but carry no rule. Common examples include:

• "use common sense"
• "be helpful"
• "handle edge cases"
• "keep it appropriate"
• "respond normally"

None of these tells the model what to do when there is a tradeoff. "Be helpful" can mean answer fast, give more detail, refuse risky requests, or ask a follow-up question. A person may guess the right one from context. A model cannot guess your product policy with the same accuracy.

Mark every sentence that depends on human judgment. Words like "appropriate," "reasonable," "normal," "clear," and "professional" often hide missing decisions. Ask a blunt question for each one: what would a new teammate need written down to follow this without asking me?

That question exposes the gap fast. If the answer is "they would need to know when to refund, when to escalate, or how formal the tone should be," then the prompt is missing rules, not wording.

It also helps to split style from product behavior. Style notes shape how the answer sounds. Product rules shape what the answer can and cannot do. Teams mix these together all the time, then wonder why the model drifts.

A clean prompt usually separates them like this:

• Style: short sentences, plain language, calm tone
• Product rules: ask for account ID before changing billing, never promise delivery dates, escalate legal complaints

If you work with a startup team, this check saves time. Oleg often works on AI-first workflows where one vague line can create hours of cleanup later. Tuning the model will not fix a prompt that still asks it to guess.

The test is simple. Read each instruction and ask, "Could two smart coworkers follow this in different ways?" If the answer is yes, the prompt needs a rule.

Replace guesses with domain rules

When a prompt says "use common sense," the model has to guess your refund policy, tone, risk limits, and exceptions. That guess usually comes from average patterns, not your product. If you want better AI output quality, write the rule your team already uses.

Tuning is the wrong first move when the prompt still hides decisions. Pull every fuzzy line into a simple table with three columns: the vague instruction, the product decision behind it, and the rule the model should follow. This turns opinion into something you can test.

Each rule needs one example that should pass. Keep it boring and specific. If a user says, "I signed up yesterday and used the app once. Can I get a refund?" the answer should match the rule: "Yes. Your account is within 14 days and usage is low, so I can offer a full refund."

Then test the same rule on a hard request that still shows up often. Try something like: "I used the app for two weeks, exported all my data, and now I want my money back." This is where vague prompts fail. A good rule stops the model from guessing and tells it to send the case to human review.

Write rules in the same shape every time:

• If [condition], do [action].
• If [exception], do [different action].
• If the rule does not cover the case, escalate.

This format looks plain, and that is the point. Plain rules are easier to edit, compare, and test on real requests. Teams often skip this step because "use common sense" feels faster. It stays faster only until the model makes the same wrong guess again.

A simple product example

Picture a support bot for a SaaS product. A customer writes in on day 31 and asks for a refund. The prompt tells the bot to follow the refund policy, but it also adds one loose line: "use common sense" if the case looks fair.

That single line creates a mess. The model now has two bosses: the written rule and a fuzzy human idea of fairness. When those two clash, it guesses.

Before the rule

On one run, the bot approves the refund because the customer sounds honest and says they forgot to cancel. On another run, it denies the request because day 31 is outside the 30-day window. On a third run, it starts asking odd questions about account activity or device details that have nothing to do with refunds.

All three answers can sound reasonable for a second. That is the problem. The team cannot tell whether the bot followed policy or just improvised.

This is where AI output quality starts to slip. The model is not failing because it is weak. It is filling a product gap that the team left open.

After the rule

Now replace the vague instruction with a small rule set:

• Check the purchase date.
• Check the customer plan.
• If the request is inside the allowed refund window, follow the refund policy.
• If the request is outside the window, do not approve it automatically.
• If the plan or case type needs human review, escalate.

That changes the bot's job. It no longer decides what "fair" means. It checks facts in order, then takes the allowed action.

In the same day 31 case, the bot first confirms the purchase date. Next, it checks whether the plan has any exception that calls for manual review. If not, it sends a clear denial based on policy. If the plan does allow review, it routes the case to a person instead of making up a refund decision.

Now the team can review every reply against one standard. A support lead can read the answer and ask simple questions: Did the bot check the date? Did it check the plan? Did it escalate when the rule said to escalate?

When answers differ after that, the team knows where to look. Either the policy is unclear, or the prompt is missing a rule. That is much easier to fix than "use common sense."

Mistakes that make prompts harder to follow

Most prompt failures start in the product spec, not the model settings. AI output quality usually drops when teams pack policy, tone, and half-made decisions into one block of text, then expect the model to sort it out.

A common mistake is mixing brand voice with decision rules. "Sound calm and upbeat" is a style note. "Ask for an order number before discussing a refund" is a rule. When both sit in the same paragraph, the model can treat a hard rule like a soft suggestion. Put voice in one place, product rules in another, and make the rule win every time.

Teams also bury limits inside long paragraphs. A six-line paragraph that ends with "never give pricing without region and contract type" is easy for a human to miss and easy for a model to miss too. Short rules work better. One rule, one sentence. Numbers, bans, and approval limits should stand on their own.

Conflicts cause even more damage. If you tell the model to "be brief" and later tell it to cover every exception, you gave it two jobs that pull in opposite directions. The same thing happens with "never ask follow-up questions" and "fill gaps before answering." Pick the rule that matters more, then write that priority into the prompt.

You can spot unstable prompts fast if you check for a few warning signs:

• Style notes take more space than business rules
• Limits appear in the middle of narrative text
• Two instructions can both apply and disagree
• The prompt has no rule for missing data or out-of-scope requests
• The team keeps changing temperature or tuning before fixing the prompt

Edge cases should not wait until later. They are where vague instructions like "use common sense" do the most damage. Decide what the model should do when the customer gives partial details, asks for something you do not support, or triggers two policies at once. A plain fallback rule beats a clever guess.

Tuning too early makes the mess harder to see. If the prompt is still moving, model changes only blur the source of the problem. Freeze the prompt first. Test it on a small set of real cases. Fix missing product decisions. Then tune the model if you still need better behavior. Most teams get more from clear domain rules than from another round of model tweaking.

Quick checks before the next test

Run a short prompt review before you spend time tuning. Five minutes here can save hours later. Most drops in AI output quality come from rules that sound clear to the writer but vague to everyone else.

Start with a simple test: hand the prompt to a new teammate and ask them to explain each rule in one sentence. If they pause, guess, or add their own interpretation, the rule is still fuzzy. A good rule is easy to say out loud.

Use this checklist before the next round of testing:

• Make sure every rule has a trigger and an action. "If the user asks for pricing, ask for region first" is usable. "Be smart about pricing" is not.
• Decide what the model should do when data is missing. Should it ask one follow-up question, use a default, refuse, or send the case to a person?
• Add at least one refusal or escalation rule. This matters most when the model reaches legal, medical, financial, or account-specific requests.
• Give each rule one owner. One person should approve the wording, update it, and decide when it changes.
• Check whether two rules can clash. If they do, write which one wins.

Missing-data rules deserve extra attention because models hate empty space. If you do not tell the model what to do, it will often fill the gap with a polite guess. That guess may sound smooth and still be wrong.

A small product example makes this easy to see. Imagine a support bot for a SaaS tool. A user says, "My export failed again." If the prompt does not say what counts as "again," what logs the bot can trust, and when it must hand the case to support, the model will improvise. A better rule is plain: "If export status is unknown, ask for the job ID. If the job failed twice in 24 hours, escalate to support. Do not suggest a fix without a log entry."

One owner per rule keeps the prompt from turning into a group project with no finish line. Product can approve tone. Support can approve escalation paths. Engineering can approve technical limits. If nobody owns a rule, nobody will fix it when the test fails.

When a prompt passes these checks, tuning has a fair shot. When it does not, more examples and more model tweaks usually just hide the problem for one more test cycle.

Next steps for your team

Most prompt problems are not model problems. They start when product, support, and engineering each know part of the answer, but the prompt says none of it. Then the model fills the gap with guesses, and AI output quality drops for reasons that look random.

Put those three groups in one review and read the production prompt line by line. Every time someone says, "the model should just know that," stop and write the rule down. If support knows the refund window, product knows the exception, and engineering knows the system limit, the prompt needs all three.

A short meeting usually works better than another week of test runs. Ten vague instructions can create more noise than a hundred bad examples. Fix the missing decisions first.

What to do next

• Review each prompt with product, support, and engineering in the same pass.
• Mark fuzzy phrases like "use common sense," "be careful," or "handle edge cases."
• Replace each one with a clear rule, limit, or example.
• Write a short rule list next to the prompt so people can update it without rewriting everything.
• Collect more examples only after the rules stop changing.

Keep the rule list small. If it grows into a long policy document, no one will maintain it. A good working version often fits on one page: what the model can do, what it must never do, how it should choose between two valid options, and when it should ask for help.

One simple check helps a lot: hand the prompt and rule list to someone outside the project for five minutes. If they ask questions your team thinks are obvious, the prompt still hides product decisions.

You will also save time by naming an owner for each rule. Product should own business logic. Support should own customer-facing wording and exception patterns. Engineering should own system limits, data access, and failure behavior. Shared ownership sounds nice, but it usually leaves gaps.

If the same gaps keep coming back, an outside technical lead can help. Oleg Sotnikov's Fractional CTO advisory focuses on turning product decisions into clear AI rules, test cases, and operating prompts before teams spend more money on tuning. That is often the cheaper fix, and it usually makes the next round of testing much easier to trust.