How to choose an AI chatbot for your website

A useful website chatbot does not merely answer questions. It moves a visitor toward a sound decision, knows when it lacks authority, and transfers the conversation to a person without making the visitor repeat the story. If it cannot do those jobs reliably, fluent prose only makes the failure harder to spot.

The choice between building and buying therefore starts with the outcome and the risk, not with a model demo. Buying usually wins for an initial support or lead capture launch with a narrow scope. Building becomes rational when the conversation depends on proprietary workflows, unusual permissions, tight unit economics, or a handoff that affects revenue. Many companies need a bought interface with a small custom control layer between the vendor and their systems. Treat that hybrid as a valid option, not an embarrassed compromise.

I have watched teams spend weeks polishing answers while a prospect with serious buying intent sat in an unowned inbox. The expensive part was not inference. It was the missing operating design: nobody had defined which questions the bot could answer, which actions it could take, who received an escalation, or how anyone would prove the system improved sales.

How to define the capability you actually need

Start with the hardest promise the chatbot must keep. A bot that retrieves opening hours has a different risk profile from one that recommends a plan, changes an order, or qualifies a large opportunity. Calling all four an AI chatbot hides the decisions that set cost and liability.

I use four capability tiers. Tier 1 publishes approved answers from a controlled knowledge base. It can cite the source, collect contact details, and refuse questions outside its scope. Tier 2 retrieves account or product data after the visitor authenticates, but it cannot change anything. Tier 3 invokes tools to create tickets, update records, schedule meetings, or modify an order. Tier 4 pursues a business goal across several systems, choosing steps and recovering from failures with limited supervision.

Every move up a tier adds more than model capability. It adds identity checks, permissions, tests, logs, failure handling, and an owner who can stop the system. A Tier 1 bot may be a publishing feature. A Tier 3 bot is production software with a conversational front end. A Tier 4 bot is an operational actor, and most websites do not need one.

Write the capability boundary as verbs before looking at products:

• Answer from approved public material and show the source title.
• Ask qualification questions and create a lead only after consent.
• Read subscription status for an authenticated user.
• Never quote custom pricing, approve a refund, or change access.
• Transfer sales, billing, and safety questions with a complete transcript.

This short contract prevents demo gravity. Vendors will show whatever looks most impressive, and internal engineers will reach for whatever is interesting to build. The verbs force both groups to demonstrate the same job. They also expose whether you need generative AI at all. A small decision tree often handles routing and consent more predictably, while retrieval and generation handle the long tail of questions.

Why buying wins the first release

Buy the first release when your differentiation is the business behind the chat, not the chat infrastructure itself. A competent service can supply the widget, message storage, knowledge ingestion, basic analytics, operator inbox, and integrations that would otherwise consume months of engineering and product work. You learn about real visitor behavior while the scope is still cheap to change.

Buying is especially sensible for Tier 1 and modest Tier 2 use cases. The required features are common, and the hidden work is substantial: browser compatibility, mobile layout, accessibility, rate limits, attachment handling, abuse controls, transcript export, and operator presence. None of that makes your company special. All of it can break a launch.

Do not confuse buying with outsourcing accountability. You still decide what data enters the service, how long transcripts remain, which sources count as approved, and who handles an escalation. You also need an exit path. Export a sample transcript and knowledge set before signing, document every integration, and confirm that you can disable the bot without taking the rest of the website down.

A bought system becomes a poor fit when its permission model cannot express your rules, its retrieval cannot separate tenants, its event data cannot support your funnel analysis, or its handoff works only inside an inbox your team does not use. Those are structural mismatches. A longer prompt will not repair them.

The popular recommendation to compare vendors by answer quality alone is wrong because a short demo rewards fluency and forgives operations. Put each candidate through the same awkward cases: a stale policy, a request to reveal another customer's data, a visitor who changes intent halfway through, an offline sales team, and a tool call that times out after the external system completed the action. The winner is the system that fails visibly and recoverably, not the one with the warmest greeting.

When building earns its cost

Build the control layer when the chatbot participates in a workflow that your company understands better than any vendor can. This does not mean training a model or writing a chat widget from scratch. It usually means owning orchestration, permissions, retrieval rules, event schemas, and handoff behavior while renting models and ordinary interface components.

Four conditions make that investment defensible. First, the bot touches proprietary actions that create revenue or liability. Second, permission rules vary by tenant, role, region, or contract. Third, conversation volume makes vendor pricing materially worse than your measured operating cost. Fourth, the conversation itself is part of the product experience and you need experiments a packaged service cannot run. One condition may justify a thin adapter. Several can justify a dedicated system.

The architecture should keep the model away from direct authority. Let the model propose an intent and arguments. A deterministic service checks identity, policy, required fields, rate limits, and current state before it calls a business system. The same service records the result. The model never receives a general database credential and never decides whether its own request is authorized.

A minimal boundary can look like this:

action: schedule_sales_call
allowed_for: anonymous_visitor
requires:
  - explicit_contact_consent
  - email
  - timezone
limits:
  attempts_per_session: 2
on_failure:
  create_handoff: true
  expose_internal_error: false

This artifact does more work than a page of prompt instructions. Engineers can enforce it, reviewers can challenge it, and tests can cover it. If a product cannot express the boundary outside a prompt, it has not earned permission to take the action.

Full custom development is still a bad default. Teams routinely underestimate operator tooling, content governance, redaction, evaluation, and incident response. Own the layer where your rules differ. Rent the rest until a measured constraint forces a change.

How to calculate the total cost curve

Compare annual cost across the expected conversation range, not a single vendor quote. The curve matters because fixed engineering cost favors buying at low volume, while fees charged per conversation, integration limits, and operator inefficiency can change the result later. A spreadsheet with honest assumptions beats a generic claim that building is cheaper.

Use this model for each option:

Annual cost = platform fees
            + model and retrieval usage
            + implementation labor amortized over useful life
            + integration and security work
            + content maintenance
            + evaluation and incident response
            + human handling time
            + expected cost of failures

Measure human handling time in the same sheet. A cheap bot that sends vague escalations can cost more than an expensive bot that gives an operator the issue, customer state, attempted actions, and next decision. Include the time sales spends reading weak leads and the time support spends correcting confident answers. Those costs rarely appear in a software proposal, but payroll absorbs them.

Create three volume cases: current traffic, a credible case for the next twelve months, and a stress case. For each, record conversations, messages per conversation, retrieval calls, tool calls, handoff rate, minutes per human handoff, and fully loaded hourly labor. Keep assumptions visible. If someone changes the expected handoff rate from 15 percent to 5 percent, the sheet should show exactly why the result moved.

Expected failure cost needs restraint. Do not invent a dramatic probability. List concrete failure types, use your own observed frequency where it exists, and leave unknown values as a decision risk. A wrong public FAQ answer may require a content correction. A duplicated refund or leaked account detail has a different consequence. Separate them.

The point where costs match is not a command to migrate. Rebuilding introduces transition cost, regression risk, and a second operating system for a while. Set a trigger such as two consecutive quarters above a cost threshold, or a blocked business requirement with documented revenue impact. Then review the decision. This keeps a theoretical saving from turning into a premature platform project.

Which data and security boundaries matter

The chatbot should receive the least data needed for the current turn, for the shortest useful period. Website conversations attract personal details, account identifiers, confidential plans, and pasted secrets even when the input box asks visitors not to share them. Design for what people will type, not what the placeholder requests.

Separate four stores: raw conversation, structured customer facts, retrieved source passages, and operational audit events. They have different readers and retention needs. A sales representative may need the transcript, an analyst may need only event names and outcomes, and an evaluator may need a redacted sample. Putting everything into one searchable transcript index makes access easy until you need to enforce deletion or investigate exposure.

For an authenticated bot, resolve identity outside the model and pass a temporary internal subject identifier. Retrieval must apply tenant and role filters before returning text. Filtering the model output afterward is too late because the model has already received the forbidden material. Tool execution needs the same authorization on the server that any other interface uses. The fact that a request arrived through natural language does not lower the standard.

The NIST AI Risk Management Framework separates measuring risk from managing it. That distinction is useful here. An evaluation can reveal that the bot sometimes discloses system instructions, but management means reducing accessible data, blocking unauthorized tools, monitoring attempts, assigning an incident owner, and testing the fix. An adversarial test report without an operating response is a document, not a control.

Require answers to carry source identifiers internally even if the interface shows friendly source titles. Record the knowledge version, model version, policy decision, tool arguments after redaction, tool result, and handoff reason. Do not log hidden reasoning or collect extra sensitive text for imagined future analysis. Logs should let you reconstruct an action without becoming a richer target than the systems they describe.

Before launch, settle retention, deletion, export, subprocessor review, regional requirements, breach handling, and who can read transcripts. Legal language varies by company and jurisdiction, so a template cannot decide it. The engineering requirement is simpler: every policy must map to a storage location, an access rule, and a tested deletion path.

How to make human handoff save the deal

A handoff succeeds when the right person receives enough context to continue, within a time the visitor understands. Merely opening a ticket is not a handoff. It is a queue insertion, and it often discards the urgency and buying signal the conversation just uncovered.

Define triggers in business terms. Explicit requests for a person always qualify. So do unsupported pricing commitments, contract changes, account security, repeated retrieval failure, strong purchase intent, user frustration, and any action whose authorization is unclear. Let the model suggest a trigger, but use deterministic rules for obvious phrases and failed system events.

The transfer payload should be a stable object, not a generated paragraph that changes shape every time:

handoff_id: h_8f31
route: enterprise_sales
priority: high
reason: custom_contract_and_security_review
visitor:
  name: provided
  email: consented
  company_size: 120
summary:
  goal: replace current support platform
  deadline: this_quarter
  blocker: needs_data_residency_confirmation
attempted_actions:
  - knowledge_search:no_approved_answer
transcript_ref: conv_72ac
owner_status: paged
visitor_expectation: reply_within_one_business_hour

The values are illustrative, but the fields should be explicit. Route and priority tell the system what to do. Reason and summary help the person respond. Attempted actions prevent duplicate work. Consent records why contact details may be used. The visitor expectation prevents the fake immediacy of a chat box when nobody is available.

Design the offline path with the same care. State that a person is unavailable, collect the minimum contact information, offer a realistic response window, and let the visitor copy the transcript or receive a reference. Never loop back to the bot after it has admitted the request needs a person. Never ask the visitor to repeat answers already present in structured fields.

Ownership needs an escalation clock. If the first route does not accept the conversation, send it to a backup or change the promise shown to the visitor. Sales and support leaders should review abandoned and bounced handoffs weekly. A chatbot can increase leads while reducing closed deals if it adds friction at precisely the moment a buyer asks for certainty.

How to keep the knowledge trustworthy

Knowledge quality needs an owner, an approval state, and an expiry rule. Retrieval cannot rescue contradictory pages, abandoned release notes, or a sales deck that promises something the current contract excludes. The model will often combine those fragments into a smooth answer, which makes a source problem look like an intelligence problem.

Create a source registry before ingestion. For every collection, record its owner, intended audience, authority, sensitivity, update mechanism, and maximum age. A current policy page should outrank an old support transcript. A signed customer agreement should govern that customer but must never become a general answer source. Marketing copy can explain benefits, but it should not authorize a refund or security commitment.

Do not dump the entire company drive into an index. Start with the smallest approved corpus that answers the launch cases. Remove duplicate exports, navigation fragments, drafts, comments, and pages with no accountable owner. Split content on semantic boundaries such as a policy clause or a complete procedure, then keep the document title, section, version, audience, and effective date as metadata. Blind chunks of equal character length often separate a condition from its exception.

Conflicts need a deterministic response. If two authoritative sources disagree, the bot should withhold the disputed claim, identify the conflict internally, and hand the case to the content owner or relevant business team. Asking the model to choose the passage that sounds newer is not governance. Dates can be missing, documents can be copied, and contractual exceptions can look less polished than public copy.

Give operators a correction path that does not silently rewrite history. They should flag an answer, attach the correct source, and send it for approval. Once approved, publish a new knowledge version and rerun affected evaluation cases. Keep the original conversation linked to the version that produced it, so an investigation can reproduce the answer instead of testing against today's different corpus.

Freshness should follow consequence. Opening hours may tolerate a scheduled check, while current availability, account state, or a regulated disclosure may need a live system read. Do not solve rapidly changing facts by indexing them more often if an authoritative API already exists. Retrieval is for knowledge; tools are for current state. Blurring those jobs creates answers that were accurate when indexed and wrong when shown.

Measure source health alongside answer outcomes. Track unanswered questions, retrieval with no approved result, citations to expired material, conflict events, and repeated operator corrections. Assign each issue to the source owner, not to an anonymous AI backlog. A chatbot exposes the quality of company knowledge, but it cannot own that knowledge.

How to run the chatbot after launch

Production ownership must exist before traffic arrives. Name one business owner for outcomes, one technical owner for availability and controls, source owners for content, and queue owners for every handoff route. If these responsibilities live in a general innovation group, failures will bounce between departments while visitors wait.

Set operating thresholds that trigger action. Examples include a spike in requests with no approved source, a drop in handoff acceptance, repeated tool timeouts, a rise in prohibited claims, or an unusual volume from one session. Thresholds should open a specific response: disable an action, narrow traffic, revert a knowledge version, switch the bot to a mode that only answers, or remove it from the page. An alert without a safe response only tells you that damage continues.

Prepare three switches. The first stops an individual tool while preserving ordinary answers. The second disables generation but keeps deterministic contact and routing forms available. The third removes the chat entry point. Test them in production with authorized staff and record who can use them. A global switch hidden in a vendor administration page is not enough if only an absent contractor has access.

Review conversations by risk and outcome, not by random curiosity. Sample all serious policy failures and customer data events, then stratify the ordinary sample by intent, source, language, route, and result. Protect reviewer access and redact exports. Reading transcripts can improve the system, but casual browsing turns visitor disclosure into internal entertainment and creates another privacy problem.

Changes need release discipline. Version prompts, policies, source collections, model settings, tool contracts, and routing rules. Run the relevant evaluation slices, record approval for risky changes, deploy to a narrow traffic segment, and compare downstream results. Roll back the smallest changed layer when behavior regresses. Updating five layers together may move a metric, but it will not tell you which change caused it.

Set a monthly business review around decisions. Which intents deserve new approved content? Which handoffs arrived at the wrong team? Where did operators override summaries? Which pages produce qualified conversations? Which automated action should lose or gain scope? The review should end with owners and due dates, not a gallery of amusing transcripts.

Plan the exit while the system works. Export conversation records in a usable format, keep your source registry independent of the vendor, document tool contracts, and avoid embedding business rules only in a proprietary visual editor. Test an export and the fallback used when the widget is disabled once a quarter. Portability is not an abstract procurement concern. It determines whether you can leave after pricing, product direction, or risk tolerance changes.

How to evaluate the bot before real traffic

Evaluate business behavior, retrieval, safety, and handoff separately so a strong score in one area cannot hide a failure in another. One blended accuracy number is almost useless. A bot can answer common questions well and still fail every important transfer.

Build the test set from real material: search queries, sales objections, support tickets, policy exceptions, misspellings, and questions your content cannot answer. Label the approved answer or action, allowed sources, disallowed claims, required tool, expected handoff route, and severity if wrong. Include cases spanning several turns because permissions and intent often change during a conversation.

A compact evaluation record might contain:

case_id: pricing_014
input: Can you guarantee this price for our European subsidiary?
expected: handoff
required_route: enterprise_sales
forbidden: quote_binding_price
required_context: region, company, requested_plan
severity_if_wrong: high

Run this suite against every vendor and every meaningful prompt, model, retrieval, policy, or content change. Review failures by slice: intent, language, customer type, knowledge source, tool, and handoff reason. Passing aggregate results can conceal a complete failure for one region or one expensive workflow.

For retrieval, grade whether the system selected an approved and current source before judging prose. For actions, grade authorization, argument correctness, idempotency, and recovery from timeouts. For handoffs, grade route, payload completeness, operator acceptance time, and whether the visitor got an honest expectation. Human reviewers should inspect severe cases even when an automated grader approves them.

Launch behind a traffic percentage or a narrow page group. Watch unanswered intents, source misses, handoff acceptance, repeated questions, action failures, and downstream outcomes such as qualified meetings or resolved requests. Conversation count and containment rate are not enough. A high containment rate may mean the bot prevented people from reaching help.

How to choose without a beauty contest

Use a weighted decision record tied to the capability contract. Weight business outcome, handoff, security boundary, integration fit, evaluation access, operating effort, and cost under each volume case. Score evidence from tests, not promises from a sales call. Any mandatory control should be pass or fail rather than something a pretty interface can offset.

Establish a manual baseline before judging automation. Record how many visitors currently reach the right person, how long the first useful response takes, how much employee time each qualified conversation consumes, and what happens afterward. Use the same outcome window for the pilot. If the bot creates more booked meetings but those meetings have lower qualification or attendance, the extra volume may not help. Compare against pages without the bot when traffic permits, and annotate campaigns or product launches that change visitor intent. This baseline also disciplines the debate over building or buying: a proposed feature earns priority when it improves a measured bottleneck, reduces a named risk, or removes proven manual work. Curiosity is a fair reason for a small experiment, but it is not a reason to grant production access or commit to an annual platform contract.

A practical selection sequence has five moves:

1. Freeze the verbs for the initial release, forbidden actions, data classes, and handoff routes.
2. Give bought, hybrid, and built options the same test cases and cost model.
3. Run a pilot with a fixed duration on pages with low risk, with real operators receiving transfers.
4. Review failures and downstream outcomes with sales, support, security, and engineering owners.
5. Record the choice, rejected alternatives, assumptions, and a date or metric that triggers review.

The hybrid option often survives this process because it preserves speed without handing policy to a prompt. A vendor can own the widget and ordinary inbox while your service owns identity, allowed retrieval, tool authorization, events, and routing. Keep that service small. If it starts reproducing the vendor's transcript viewer and content editor, the boundary has drifted.

A Team & AI Audit at oleg.is can map this decision against engineering capacity, payroll, and the workflows that deserve automation before a company commits to a platform or a custom build. The useful deliverable is the operating model and cost case, not another enthusiastic chatbot demo.

Pick the lowest capability tier that closes the customer gap. Make the handoff contract part of the architecture, put human time into the cost curve, and require every option to survive the same failure tests. If nobody owns the conversation after the model stops, the website does not have a chatbot system. It has an animated delay before a person eventually discovers the lead.