Reduce AI assistant hallucinations with shorter paths

Why assistants start guessing

An assistant gives its best answers when it can reach the exact source it needs right away. That source might be a live product spec, a customer record, a policy document, or the one folder where your team keeps the real final version of a file. When the model can read that source directly, answers are usually clear. When it cannot, it starts filling gaps.

People call that hallucination, but the behavior is less mysterious than it sounds. The model is not deciding to invent things. It is predicting the most likely answer from the clues it has. If the path to the truth is slow, messy, or blocked, wrong guesses become more likely.

That usually happens because of extra hops. A file lives in one system, access lives in another, and approval depends on someone replying later. By the time the assistant has enough context, it has already spent most of its effort searching, waiting, or working from partial information. That is when answers turn vague and actions drift off course.

You can spot the problem early. Simple requests take too long. Replies sound generic or overly cautious. The assistant picks the wrong file, tool, or next step.

The causes are usually mundane. Retrieval is too broad, so the assistant gets ten near matches instead of one clear source. File access is patchy, so it can read old docs but not the current one. Approval rules add delays at the worst moment, even when the action is harmless.

Many teams start by rewriting prompts. That can help a bit. It does not fix a long route to the truth. If the model has to hunt across folders, ask for access, and wait for approval before it can act, guessing is the expected result.

The pattern is simple: short path, better output. Long path, more invented detail.

Where the long route hides

A model rarely guesses out of nowhere. It guesses after it fails to reach the real source fast enough.

The trouble often starts when the answer lives in one place, but the assistant can only reach a copy of a copy. Someone asks about pricing, a policy, a deployment step, or last week's decision. The assistant checks a retrieval layer. That layer checks an index. The index depends on a sync job. The sync job pulled from a folder that nobody has updated in a month.

Hidden hops that cause trouble

The path between a question and the truth often looks like this:

• chat tool to retrieval service
• retrieval service to search or vector index
• index to synced documents or exported pages
• copied docs to original files, tickets, or code
• assistant to a person for missing access or approval

A direct path is simple. The assistant reads the source file, the live ticket, the current database schema, or the approved policy document. A relay path is much messier. The assistant reads a summary tool that reads a knowledge base that mirrors a drive that mirrors the real working folder. Every relay can change wording, drop context, or lag behind.

Stale copies create one kind of confusion. Missing permissions create another. If the assistant can see the project wiki but not the repository, it may answer a code question from old docs instead of current code. If it can open design notes but not the billing system, it may invent a status because the real number sits behind a blocked tool.

Every extra step adds two costs: delay and doubt. Delay is obvious. More tools mean more calls, more waiting, and more places to fail. Doubt is quieter, but it does more damage. The model has to decide whether the partial answer it found is good enough. On a long route, it sees fragments instead of facts.

That is why some teams blame the model when the real problem is access design. If the shortest route to the truth takes six hops and two approvals, the assistant spends more time patching holes than answering the question.

How to shorten retrieval

Most retrieval setups are too wide. Teams dump every handbook, ticket export, and old wiki into one index, then expect the assistant to pull out one clean answer. Usually it cannot. The model sees too much, gets mixed signals, and fills the rest in.

Cut the search space first. Start with the questions people ask every week. A support assistant might need pricing rules, refund policy, and account states. An engineering assistant might need the current runbook, system map, and release notes. It does not need five years of archived docs in its default path unless those archives answer live questions.

Smaller retrieval sets often work better than bigger ones. Keep the sources that directly answer common tasks. Move the rest out of the default flow. Older material can stay available for manual lookup.

A useful rule is this: one question should map to a short list of trusted sources. If the assistant has to search multiple indexes, open a summary page, and then follow a trail to the source file, the route is already too long.

Chunk size matters too. Large chunks look efficient, but they often hide the one line that actually answers the question. Shorter chunks with clear labels work better. The labels do not need to be fancy. The source name, owner, last update date, document type, and version are usually enough.

Duplicates cause quiet damage. Teams often index the same policy in a PDF, a wiki page, a copied document, and a ticket comment. Once one copy changes and the others do not, retrieval stops returning truth and starts returning conflict. Pick one canonical source for each topic. Remove stale copies and old indexes.

Measure the path, not just the final answer. Count the hops from question to source. One or two hops is usually fine. Beyond that, guessing starts creeping in. You see this quickly in software teams: if a coding assistant needs the current deployment rule, it should read the live repo docs or runbook directly, not a summary of a summary from an old workspace.

Short retrieval paths are not fancy. They are strict. That discipline saves time and keeps answers grounded.

How to fix file access

Bad file access makes an assistant guess. Good file access lets it read the real source, in the real place, when it needs it.

If the task is "update the onboarding email," the assistant should open the current template, the shared copy document, and the place where that email is triggered. A pasted excerpt is often not enough. Tiny gaps turn into invented details.

Structure matters as much as permission. Group files around the jobs people actually do, then split them by sensitivity. Product specs, support macros, and release notes can live together. Payroll, legal records, and private customer data should stay separate. That keeps routine work simple while still protecting sensitive material.

Names matter too. When a team stores files called "final," "final v2," and "new final," both people and assistants pick the wrong one. Use plain names. Add dates when needed. Keep one current version that everyone treats as the source.

Write rules should match the work. Some teams give broad read access but block every edit, so the assistant can inspect a problem but cannot fix the file that caused it. Others allow writes in messy shared folders. A better setup is narrower and clearer:

• let the assistant read the source files tied to its task
• allow writes only in approved folders or branches
• keep sensitive folders separate from everyday work
• archive old versions so the current one is easy to find

Logs close the loop. Record which files the assistant opened, which ones it changed, and who approved the change when approval was required. That trail helps when something goes wrong. It also shows where the route to the truth is still too long.

This alone can save a product team hours each week. Instead of asking the assistant to work from pasted notes in chat, let it open the live spec, the current UI text, and the test file in one workspace. The model stops guessing because the blanks are gone.

How to trim approvals

Too many approval steps make an assistant slow and unsure. When it cannot read a file, inspect a diff, or run a safe check without asking first, it starts working around missing information.

Most teams approve far too much. Reading files, searching code, drafting a reply, summarizing logs, and suggesting a patch usually do not need human permission every time. Changing production data, using secrets, sending messages to customers, or deleting something does.

A good rule is simple: approve writes, risky external actions, and anything that is hard to undo. Let low risk reads and drafts happen by default.

Small requests matter here. Do not ask someone to approve a vague action like "update the config." Ask for one clear step with enough detail to judge quickly:

• the exact file or system the assistant wants to touch
• the diff or command it plans to run
• the reason in one sentence
• a simple "yes" or "no" choice

That format cuts delay because the reviewer does not need to investigate first. They can decide in seconds.

Time limits help too. An approval that sits for two hours is almost as bad as a rejection because the assistant loses context and may start down a different path. Set an expiry time, then define what happens next: retry, ask a different person, or stop cleanly.

In practice, a tiered model works well. Let the assistant read logs, inspect CI output, search the repo, and draft code without interruption. Make it ask before it changes infrastructure, uses credentials, merges to protected branches, or touches customer data.

The goal is not fewer controls at any cost. The goal is sharper controls. When approvals are rare, specific, and fast, the model spends more time working from facts and less time improvising around blocked paths.

A simple example from a product team

A product team had a support assistant that answered refund questions. Customers asked basic things like "Can I get my money back?" or "Why was my refund denied?" The assistant should have handled most of them, but it kept guessing because the path to the facts was too long.

The old flow was messy. First, the assistant searched a help center index that mixed current policy pages with old copies. Then it pulled notes from an internal wiki where support leads had pasted exceptions over time. If the order looked unusual, the assistant sent the case into a manager queue, even for small refunds.

That setup wasted time and produced contradictions. One document said refunds were allowed within 14 days. Another said 30 days for annual plans. A copied note mentioned a holiday exception that had expired months earlier. Agents kept stepping in because the assistant quoted the wrong rule or asked for approval when none was needed.

The team cut the path down to two sources. The assistant could read one live policy file, owned by finance and support together, and one customer record with the order date, plan, payment status, and refund history. Nothing else counted as ground truth.

They also added one approval rule that anyone could understand. If the refund was under $50 and the purchase was within the policy window, the assistant could approve it. If it was $50 or more, outside the window, or the account had abuse flags, the assistant sent it to a manager.

The change showed up fast. Before, the median answer took about 8 minutes because the assistant searched, compared, and stalled in queues. After the change, most replies went out in under 90 seconds.

Mistakes dropped too. Before, about 15% of refund answers needed a correction from a human agent. After the team switched to one live policy file and one customer record, that fell to about 2%.

The lesson is plain. The model did not need cleverer wording. It needed a shorter path to the truth.

Mistakes that keep the route long

Teams often add friction without adding truth. An assistant guesses more when it has to pass through too many layers, too many folders, or too many permissions just to answer a basic question.

One common mistake is the giant index. Teams dump every document, ticket, note, and transcript into one search system and hope retrieval will sort it out. Usually it does not. The assistant pulls ten loosely related chunks, misses the current spec, and fills the gap with a tidy guess.

Another problem is source material disappearing behind summaries of summaries. A product brief turns into a project summary, then a team update, then a chatbot note. By the time the model reads it, the original detail is gone. Dates vanish. Edge cases vanish. Small wording changes vanish too, and those small changes often decide whether the answer is right.

Approval flows cause the same kind of delay when nobody questions them. If an assistant needs permission to read a harmless spec, a changelog, or a test result, the tool stops so often that people start bypassing it or asking vague questions instead. Gates should protect secrets, spending, and production changes. They should not block routine reading.

Folder hygiene causes quieter damage. Old files and current files sit side by side with names like "final," "final-v2," and "current-new." The assistant sees all of them as possible truth. If your team cannot tell which file wins in ten seconds, the model cannot either.

Ownership is the last missing piece. Every source needs a person who keeps it current or retires it. Without an owner, stale docs stay around for months and still look official.

A setup usually needs work if search returns lots of near duplicates, summaries outrank source material, staff approve harmless reads by habit, version names depend on memory, or nobody can say who maintains a document.

The fixes are usually small. Keep the live source easy to reach. Archive old material clearly. Remove gates that protect nothing. Shorter paths leave the assistant less room to invent.

Quick checks for your setup

If an assistant needs three searches, two approvals, and a message to a coworker before it can read the real source, it will fill gaps on its own. Better prompts will not solve that.

Use one common task as a test. Pick something ordinary, like updating a help article or answering a customer question. Then trace the path the assistant takes.

Start with a few plain questions. Can it reach the original source in one or two steps? If it has to jump through a wiki, a ticket, and a copied note, the route is too long.

Then check file clarity. The assistant should know which file is current without guessing from names like "final_v2" or "latest-new." One clear current version beats a folder full of near matches.

Approval steps need the same test. Every gate should block a real risk, such as exposing customer data, changing production systems, or spending money. If nobody can name the risk, the gate is probably there from habit.

A good setup is usually easy for people to explain too. If a teammate cannot describe the access rules in about a minute, the system is too tangled. Models struggle with the same confusion people do.

Logs tell you where things break. They should show what the assistant tried to open, which tool or permission stopped it, and when it switched from reading facts to guessing. Without that trail, teams end up arguing about the answer instead of fixing the path.

A quick review should confirm five things:

• the assistant can reach the real source in one or two moves
• one file or document is clearly marked as current
• each approval step has a named risk behind it
• a teammate can explain the rules without a long handoff
• logs show where the assistant stopped, not just the final reply

If even two of these checks fail, fix them before you swap models or rewrite prompts. Shorter routes usually matter more than clever instructions.

What to change first next week

Start with one workflow that already hurts. Pick the task where the assistant gives wrong answers often enough that someone has to fix them every week. Support replies, policy questions, product specs, and pricing answers are all good places to look. If one bad answer keeps pulling a manager back into the loop, start there.

Then map the full path from question to source to action on one page. Include every hop: search index, shared folder, document, spreadsheet, human check, final approval. If the model has to jump across too many places before it can answer, it will guess.

Make three small cuts instead of planning a full rebuild:

• drop one search step if two tools point to the same documents
• remove one stale source that keeps old rules alive
• skip one approval gate when the action is low risk and easy to reverse

You are not asking the model to think harder. You are giving it less room to fill gaps.

Test the new flow with ten real prompts from daily work. Pull them from tickets, chat, email, or document requests. Mix simple requests with annoying cases that usually create rework. For each prompt, check three things: did the assistant reach the right source, did it answer correctly, and how long did the whole path take?

A product team can do this in a week. On Monday they pick release note drafting. On Tuesday they notice the assistant checks an old changelog, a current spec, and a chat thread before anyone approves the draft. By Thursday they keep one source of truth, give direct file access to the current spec, and remove one manager approval for minor edits. The answers get shorter, faster, and more accurate.

If your team keeps circling around the same problem, an outside review can help. Oleg Sotnikov writes about this kind of systems work on oleg.is and advises startups and small teams as a Fractional CTO. The useful part is not prompt polish by itself. It is fixing retrieval, access, and approval design so assistants can work from the real source instead of guessing.