How third-party AI risk management works in practice

Vendor AI is already inside most companies, often hidden behind an ordinary product feature, an API call, or a subcontractor the buyer never reviewed. A security questionnaire completed at purchase does not control that exposure. The vendor can change its model, training policy, subprocessors, retention settings, or degree of automation while the signed PDF stays exactly the same.

Effective third-party AI risk management treats each use as a changing production dependency. You need to know where AI touches your data or decisions, set review depth according to possible harm, put specific duties into the contract, and watch for changes after approval. The process should slow a high-impact hiring model and let a low-risk meeting summarizer move quickly. Giving both the same review is expensive theater.

I have seen vendor registers that contain a company name, renewal date, and a green status cell. That tells an incident team almost nothing. When a vendor leaks prompts or silently routes data to a new model provider, the useful questions are concrete: which workflow used it, what data crossed the boundary, who relied on the output, what can be disabled, and who owns the response? Build the system around those questions.

Inventory the use, not the vendor name

An AI inventory must record each use case separately because one vendor can create several different risk profiles. A writing assistant used on public marketing copy is not the same system when support agents paste customer tickets into it. The legal entity and contract may match, but the data, affected people, failure modes, and controls do not.

Start discovery with money and traffic rather than a company-wide survey. Review accounts payable, expense cards, browser single sign-on applications, API gateways, OAuth grants, data warehouse destinations, customer support integrations, and procurement records. Ask engineering which external endpoints receive prompts or embeddings. Ask business teams which products now summarize, score, recommend, transcribe, classify, or generate. The word "AI" may never appear in the product name.

Record a stable identifier for the use, then connect it to a vendor and an internal owner. This small JSON object is enough to expose missing facts during intake and can live in a repository or governance system:

{
  "use_id": "support-reply-draft",
  "vendor": "Example Vendor",
  "business_owner": "VP Support",
  "technical_owner": "Support Engineering",
  "purpose": "Draft replies for agent review",
  "data_classes": ["customer_ticket", "account_metadata"],
  "affected_people": ["customers", "support_agents"],
  "model_providers": ["declared_subprocessor"],
  "output_action": "human_approved_message",
  "kill_switch": "disable integration token",
  "tier": 2,
  "last_reviewed": "YYYY-MM-DD",
  "next_review": "YYYY-MM-DD"
}

Do not let "no sensitive data" pass as a value. Name the allowed data classes and the prohibited ones. Record whether the vendor stores inputs, uses them for training, sends them to another model provider, or lets an administrator disable those behaviors. Unknown is a legitimate inventory value during discovery, but it must create an open action rather than quietly becoming approval.

Shadow use deserves a safe reporting path. If employees expect punishment for declaring an AI tool, they will hide it until procurement finds a charge or security finds traffic. Offer a short disclosure form and a temporary restricted state: public or synthetic data only, no automated decisions, and no external sharing while review runs. Amnesty discovers more risk than a stern policy nobody follows.

Map data flow and decision authority

The inventory becomes useful only when it shows what enters the system, what leaves it, and what the business does with the result. Model type matters less than the consequence of a wrong, exposed, or manipulated output. A large language model that suggests internal document tags may be low impact. The same underlying model that recommends which customer gets investigated can affect rights, revenue, and reputation.

Draw one plain flow for every material use: source, vendor, model or subprocessor, output store, human reviewer, and final action. Mark where prompts, uploaded files, embeddings, logs, and generated content persist. Include feedback buttons and support tickets because vendors often collect those outside the primary processing path. If the vendor cannot explain the flow, you cannot make a sound approval decision.

Separate assistance from authority. "Human in the loop" is too vague to count as a control. A human who clicks approve on hundreds of plausible outputs has little effective authority. Record what evidence the reviewer sees, whether they can change the result, how much time they have, whether rejection has a cost, and what happens when confidence is low. Human review reduces risk only when the job design supports disagreement.

Also record who is affected even if they never touch the product. Applicants ranked by a recruiting tool, callers transcribed by a contact center, and customers whose messages train a classifier all sit outside the buying team. Their exposure should drive notice, appeal, correction, and retention decisions. Procurement cannot infer those needs from a vendor's security certification.

A useful flow ends with a reversible business action. If an output directly changes access, pricing, employment, credit, safety, or legal status, treat it as consequential even when a person technically initiated the final click. If staff can test an output, compare it with source evidence, and undo the action without material harm, the residual risk may be lower. That distinction should appear in the inventory, not live in someone's memory.

Tier by possible harm before controls

Risk tiering should start with inherent risk, the exposure that exists before vendor promises and internal controls. Teams routinely blur inherent and residual risk, then approve a dangerous use because the vendor has a polished security page. Certifications and contract terms can reduce risk. They cannot change the fact that a wrong clinical, employment, or financial recommendation may seriously harm someone.

Use four tiers that trigger clear actions:

1. Tier 1 covers public or synthetic data and outputs that do not drive a material decision. The owner can complete a short self-assessment and accept standard terms.
2. Tier 2 covers internal or customer data with meaningful review before use. Security and privacy review the data flow, retention, training terms, and access controls.
3. Tier 3 covers sensitive data, external-facing generation, profiling, or outputs that materially influence people. Legal, security, privacy, and the accountable executive must approve it, with testing and an exit plan.
4. Tier 4 covers uses that can create severe safety, rights, legal, or financial harm, or uses the company cannot supervise. The default is no deployment until an executive risk owner approves a documented exception.

Score a use across a small set of dimensions: data sensitivity, decision consequence, population scale, autonomy, detectability of error, reversibility, exposure to attackers, and concentration risk. Avoid a single average that lets low data sensitivity cancel a high decision consequence. Take the highest severe dimension as the initial tier, then document why any downgrade is defensible.

Residual risk comes after you test controls. Encryption may lower disclosure risk. A trained reviewer with source citations and sampling may lower error risk. A contractual incident notice may reduce response delay. None of these controls works merely because a questionnaire says "yes." Ask for configuration evidence, test results, sample logs, or a live demonstration, depending on the tier.

Define acceptance authority by tier before a popular vendor creates pressure. A product manager can accept Tier 1. A cross-functional group should own Tier 3. Tier 4 needs a named executive who understands the possible harm and signs a time-limited exception. Without those boundaries, the loudest buyer becomes the risk owner by accident.

Make intake short enough to use

An intake process should identify the tier in minutes and reserve deep review for the uses that deserve it. A forty-page form sent to every employee creates copied answers and hidden tools. Ask branching questions that reveal exposure: purpose, data classes, affected people, output action, integrations, retention, training use, subprocessors, and whether the feature can be disabled.

Route requests through one visible queue with a service target for each tier. Fast handling is a control because teams route around an unpredictable process. Tier 1 might receive same-day confirmation. Tier 3 may need testing, negotiated terms, and leadership review. Publish those expectations so a buyer does not promise a launch date before reviewers see the use.

Require two owners. The business owner answers for purpose, affected people, and acceptable failure. The technical owner answers for integrations, credentials, logs, configuration, and shutdown. Procurement can coordinate evidence and signatures, but it should not inherit operational risk. Security cannot decide whether a biased recommendation is tolerable in a business process it does not own.

Approval must bind to a defined use and configuration. "Vendor approved" is dangerous shorthand. Write the approval as "approved for public marketing drafts with human editing, no customer data, and training disabled." A new data source, automated action, model provider, or user population reopens review. That sentence prevents feature creep better than another annual questionnaire.

Create a restricted pilot state for uncertain cases. Use synthetic or de-identified test data, a limited user group, no production write access, and a fixed end date. The pilot should answer named questions, such as whether staff detect unsupported answers or whether the vendor logs admin changes. A pilot that quietly becomes production has bypassed the very decision it was meant to inform.

Put operational duties in the contract

AI contract clauses should turn the approved use and its controls into enforceable vendor duties. A generic data protection addendum may cover personal data processing, yet say nothing about model training, output ownership, evaluation evidence, silent model changes, or a buyer's right to disable an embedded feature.

Define the service before negotiating risk language. Name the AI features, approved purposes, data categories, territories if relevant, user groups, and prohibited uses. State whether the vendor acts as a processor, independent controller, or another role for each data path when privacy law requires that distinction. Legal counsel should adapt the language to the governing law; a copied clause cannot resolve a mismatched operating model.

The contract should address at least these duties:

• The vendor may process customer data only to deliver the defined service and may not train or improve shared models with it unless the buyer gives explicit written approval.
• The vendor must disclose model providers and relevant subprocessors, give advance notice of material changes, and offer a workable objection or termination path.
• The vendor must maintain stated retention and deletion periods for prompts, files, outputs, logs, and backups, then confirm deletion at exit.
• The vendor must notify the buyer promptly of security incidents, harmful output events, regulatory inquiries, and control failures that affect the service.
• The vendor must provide enough documentation, logs, test evidence, and cooperation for the buyer to assess performance, investigate incidents, and meet its own obligations.

Add intellectual property allocation for inputs, outputs, and vendor indemnity where appropriate. Do not accept a broad right to reuse "service data" if that definition includes prompts, files, feedback, or outputs. Ask how the vendor handles infringement claims and whether contractual protection excludes the exact workflows you plan to use. Counsel should test the exclusions against the inventory rather than admire the headline promise.

Model and feature changes need a threshold. "We may update the service at any time" gives the vendor freedom to alter your residual risk without review. Require notice before a material change to the model provider, training-data approach, safety controls, retention, subprocessor chain, or output behavior. For a high-tier use, require the ability to test before rollout, remain on a prior version for a transition period, or terminate without penalty.

Audit rights should produce evidence, not ceremonial access to a report. Accept independent reports where they cover the actual service and period, but preserve the right to ask focused follow-up questions and obtain remediation status. For severe incidents or Tier 4 uses, you may need stronger inspection and cooperation rights. A small buyer will rarely conduct a full audit, but precise evidence rights still improve answers during an incident.

Negotiate exit while the seller still wants the deal. Set export formats, deletion duties, transition help, configuration handoff, and survival of confidentiality and incident cooperation. If the workflow cannot run without one vendor, record the recovery time and manual fallback. Concentration risk belongs in the commercial decision, not in a disaster plan written after the contract renews.

Verify controls in your own configuration

Vendor evidence does not prove that your tenant, integration, and workflow are safe. Most failures I see occur in the gap between a documented feature and the buyer's actual settings. An administrator leaves model training enabled, an integration token has broad scopes, or generated answers go straight to customers because the human review control existed only in a slide deck.

Test the configuration against the approval statement. Confirm retention and training settings with screenshots or exported admin configuration. Restrict identity groups and API scopes. Separate production and evaluation credentials. Log administrative changes and model calls where the product permits it. Send canary data that must never appear in another account, but do not use real secrets or personal data as bait.

For generative outputs, build a small evaluation set from the actual job. Include ordinary cases, ambiguous requests, prohibited content, prompt injection attempts, and cases where the correct response is to refuse or escalate. Record the expected behavior and review disagreements. A vendor benchmark cannot tell you whether a support draft invents a refund policy or whether a document assistant obeys instructions hidden inside an uploaded file.

Measure the control that matters to the business action. Track unsupported claims that reviewers miss, harmful recommendations that reach a user, prohibited data accepted by the interface, or automation performed without approval. Aggregate "accuracy" can conceal the failure class that carries the highest cost. For rare severe outcomes, use scenario tests and incident readiness rather than pretending a small sample gives a reliable percentage.

Check the kill switch before launch. The owner should know how to revoke a token, disable a feature, remove write access, or route work to a manual process. Test that shutdown does not destroy evidence needed for investigation. A switch that requires an absent vendor administrator or a contract negotiation is not an operational control.

Monitor change, not questionnaire anniversaries

Continuous monitoring should look for events that alter exposure, not repeat the purchase review on a calendar. NIST AI RMF 1.0 makes this point directly in Manage 3.1: organizations should regularly monitor risks and benefits from third-party resources, then apply and document controls. I agree with the continuous intent, but "regularly" needs event triggers or it degrades into an annual email.

Watch four evidence streams. First, vendor change notices should reveal new models, subprocessors, locations, retention terms, and features. Second, technical telemetry should show usage volume, data classes if observable, access changes, error patterns, and calls from unapproved systems. Third, business monitoring should capture complaints, overrides, appeals, and downstream corrections. Fourth, external signals should include vendor incidents, material legal changes, and credible research that applies to the deployed model or workflow.

Set thresholds before the graph moves. A sudden jump in usage, a new sensitive data source, lower reviewer override rates, unexplained output drift, or a changed model identifier can trigger review. The response may be investigation, tighter restrictions, rollback, or suspension. Without thresholds and owners, a dashboard only proves that someone collected data.

Monitoring depth follows the tier. Tier 1 may need owner attestation at renewal and automated discovery of new integrations. Tier 2 needs periodic configuration checks and incident review. Tier 3 needs defined quality and harm indicators, change alerts, sampling, and a scheduled cross-functional review. Tier 4 needs active oversight, tested fallback, and executive acceptance that expires.

Do not demand telemetry the system cannot produce. Record the blind spot and compensate with sampling, narrower use, stronger human checks, or a lower approval ceiling. If you cannot observe a failure that would cause severe harm, the use may exceed your ability to operate it safely. Contract language cannot manufacture observability after deployment.

Treat vendor changes as new risk decisions

A material vendor change should reopen the relevant part of review rather than restart the whole process or pass automatically. The owner compares the proposed state with the approved baseline: model, purpose, data, subprocessors, retention, autonomy, and affected population. Only changed risk dimensions need fresh evidence, which keeps the process fast enough to follow.

Consider a support platform that adds automatic reply sending. The original approval covered drafts that agents edited. The vendor calls auto-send a productivity feature and enables it during an account upgrade. Data sensitivity has not changed, but decision authority and reversibility have. If nobody catches the configuration change, customers can receive invented commitments before a reviewer sees them.

The response is not another generic vendor review. Disable auto-send, preserve configuration and message logs, sample affected replies, notify the business owner, and assess whether any customer needs correction. Then decide whether the new use requires testing and a higher tier. This walkthrough matters because many AI incidents begin as ordinary product administration, not an exotic model attack.

Contract notice supports this process but cannot replace discovery. Product release notes may omit a default change, and a business administrator may activate a feature without procurement. Compare identity, integration, and configuration data with the inventory. Give administrators a clear rule: enabling generation, scoring, profiling, or automated action requires an approved use identifier.

When the vendor changes a subprocessor or model provider, ask whether data location, retention, training rights, security controls, or incident paths changed. A famous model name is not evidence of equivalence. Your concern is the full service behavior and contractual chain. Approve, constrain, or reject the change based on that chain.

Prepare to contain failure before renewal

Every Tier 3 and Tier 4 use needs a short incident playbook before production. It should name detection sources, decision authority, vendor contacts, evidence to preserve, shutdown steps, affected groups, communication owners, and the manual fallback. Tie it to the existing security and business continuity process so the team does not invent a separate command structure during an event.

Classify AI incidents by consequence, not novelty. A confidential prompt exposed to another tenant is a data incident. A discriminatory ranking that affects applicants is a harm and compliance incident. A generated instruction that damages equipment may be a safety incident. Use existing specialists and notification rules, then add the model, prompt, evaluation, and vendor evidence those teams need.

Preserve inputs, outputs, model or feature version, configuration, reviewer actions, timestamps, and downstream decisions where law and policy permit. Do not collect more personal data in the name of investigation. Hash or snapshot relevant configuration so later vendor changes do not erase the state. Ask the vendor for its timeline and contributing factors, but build your own account from evidence you control.

Practice a vendor outage and a harmful-output event. The first tests availability and manual fallback. The second tests whether staff can stop the workflow, find affected decisions, correct them, and communicate with people who experienced harm. A tabletop that ends with "contact the vendor" has missed most of the buyer's responsibility.

Exit is a risk treatment, not a procurement failure. Trigger it when the vendor will not disclose material dependencies, cannot meet evidence duties, repeatedly changes controls without notice, or leaves residual risk above the company's tolerance. A replacement may carry its own risk, so compare transition exposure and operate a manual fallback where possible.

Give one owner the operating picture

Third-party AI governance works when one accountable leader can see the inventory, exceptions, overdue evidence, incidents, and upcoming renewals. A committee can advise, but it cannot own an unnamed task. In a startup, the accountable person may be the CTO, security lead, or privacy lead. The role matters more than the title.

Run a monthly operating review for high-tier uses and a quarterly portfolio review for the whole inventory. Focus the meeting on changed risk: new tools, expanded purposes, material vendor notices, expired exceptions, failed tests, incidents, and exits. Do not reread every green record. Decisions should produce an owner, due date, evidence requirement, and change to approval status.

Keep the evidence package proportional. Each approved use needs the inventory record, data flow, tier rationale, approval statement, material vendor evidence, contract deviations, configuration proof, test results where required, and monitoring plan. Store links or references in the record, with access controls for sensitive documents. The goal is a reproducible decision, not a large binder.

Metrics should expose control health. Track known uses with owners, high-tier uses with tested shutdown, overdue vendor changes, open exceptions, time to review, and incidents that revealed an unknown use. A count of completed questionnaires rewards motion without showing whether risk fell. If review time rises, fix routing or staffing rather than lowering the tier in secret.

Connect renewals to this operating review. Procurement should not renew a high-tier service while evidence is overdue, an exception has expired, or the owner has left. Give the accountable leader enough notice to remediate, renegotiate, or exit without an emergency extension. Automatic renewal dates belong beside risk status in the inventory because commercial timing changes your available treatments. A vendor may be technically replaceable yet operationally unavoidable three days before renewal. Six months earlier, the company can test an alternative, export its data, and compare manual fallback costs.

Founders often discover that the hardest part is not writing a policy. It is assigning ownership across engineering, legal, security, and procurement without building a slow bureaucracy. A Team & AI Audit from oleg.is can map that operating gap and identify where AI changes team cost and control requirements. The system must still work after an advisor leaves, so insist on records, owners, and repeatable decisions.

Begin with the ten vendors that touch the most sensitive data or consequential decisions. Create use-level records, assign inherent tiers, and find the missing kill switches and contract duties. That first portfolio view will be imperfect, but it will expose real decisions. A polished policy without that view will not tell you what to shut down on Monday morning.