An AI transformation leadership trial for founders

A paid trial for an AI transformation leader should test judgment on real work, not fluency with tools. Give the candidate one delivery bottleneck, one security decision, and one cost model. Then watch whether the person can turn incomplete evidence into a bounded decision that your team can execute and reverse.

The trial is a leadership test, so a polished prototype proves very little. A strong candidate defines the business constraint, changes the smallest useful part of the operating system, leaves an audit trail, and tells you where AI should not be used. A weak candidate produces a dramatic demo, quotes model capabilities, and quietly transfers risk to the engineers who will remain after the trial.

A paid trial must buy evidence

The trial should produce evidence about how the candidate leads under your actual constraints. It should not become discounted consulting, a strategy presentation, or an audition based on unpaid speculative work. Pay a fixed fee, state the time box, and make the decision rights explicit before access begins.

For most startups, five to ten working days is enough. A shorter trial rewards theater because the candidate has no time to inspect the workflow. A longer trial starts to resemble an engagement without giving either side a clean decision point. The scope should fit inside the candidate's available time without expecting nights or weekends.

Write a one page trial charter with the candidate. It should name the bottleneck, the security decision, the cost model, the people who must participate, and the evidence due at the end. It also needs exclusions. If customer data, production deployment, personnel decisions, or vendor purchases are outside the trial, say so.

Use a scorecard agreed before the work starts:

Trial outcome                    Weight   Evidence
Delivery bottleneck understood   25%      Baseline, queue map, rejected causes
Intervention shipped or tested   20%      Change record, result, rollback
Security judgment                20%      Decision note, controls, residual risk
Cost model                       20%      Inputs, formulas, scenarios, owner
Leadership behavior              15%      Team feedback, decisions, handoff

Weights are less important than committing to them early. If you invent the criteria after seeing the result, you will select for charisma or for agreement with your own preferred answer. Record what would count as a strong, acceptable, and failed result for each line.

Do not require the candidate to guarantee an operational improvement. A real bottleneck can survive a sensible intervention because a dependency does not cooperate or the baseline was wrong. Judge whether the candidate exposed that fact quickly and changed course. Guaranteed trial outcomes encourage people to choose safe, cosmetic work.

The trial fee should reflect a bounded executive assignment, not the candidate's hoped for monthly salary divided by working days. The exact amount depends on access, risk, and expected output. Pay for the work even if you decide not to hire. Founders who try to get this diagnosis free tend to attract candidates who optimize for the sale.

Put one live delivery bottleneck in their hands

Choose a bottleneck that regularly delays business value and has enough recent evidence to inspect. A release approval queue, flaky acceptance tests, slow code review, repeated environment failures, or an overloaded senior engineer can work. "Make engineering faster" cannot, because the candidate can claim progress without confronting a specific queue.

Give the candidate the last ten to twenty items that passed through the bottleneck. Include timestamps, rework, blocked reasons, and the people involved. The sample does not need statistical power. It needs enough detail to separate waiting time from touch time and recurring causes from one unusual incident.

Ask for a baseline before an intervention. A useful baseline answers four questions: What enters the queue? When is an item considered finished? Where does work wait? Which delay can this team control? If the candidate reports only cycle time, ask for the distribution or the individual items. One average can hide a handful of jobs that do nearly all the waiting.

The candidate should then propose the smallest reversible change that tests a cause. Suppose releases wait two days for a senior engineer to review routine database migrations. Buying a coding agent may produce migrations faster and make the queue worse. A better trial might classify migrations by risk, automate checks for the low risk class, require review for destructive operations, and run the rule on recent changes before enabling it.

A simple experiment record keeps the work honest:

bottleneck: release approval
baseline_window: last 15 releases
suspected_cause: routine migrations wait for expert review
change: automated checks plus risk-based review
success_signal: fewer low-risk releases waiting over one business day
guardrail: zero unreviewed destructive migrations
rollback: restore mandatory review for every migration
owner_after_trial: platform lead

The candidate may use Claude Code, Codex, or another tool during the intervention, but tool choice is not the result. Inspect how the candidate supplies context, checks generated changes, handles failure, and fits the tool into the existing repository and review process. A ten minute generation run that creates two days of cleanup is negative evidence.

Watch how the person works with the engineer who owns the queue. Good leaders ask for the owner's model of the failure, test it against records, and give that person a meaningful role in the change. Weak leaders treat resistance as ignorance or ask engineers to provide data while excluding them from the decision.

Do not choose the most politically explosive bottleneck for a trial. A candidate cannot fairly resolve a founder conflict or redesign the organization in a week. Choose a problem with real consequences and a reachable owner. You are testing diagnosis and execution, not the candidate's willingness to start a civil war.

Definition of done exposes leadership quality

A strong candidate defines the decision and the stopping condition before building anything. This habit distinguishes an operator from an enthusiastic tool user because AI can generate plausible work long after the useful question has been answered.

Ask the candidate to write a definition of done for all three trial threads. For the delivery bottleneck, "done" might mean that the team has tested one queue rule against recent work, measured the result, and assigned an owner. It should not mean that every release problem disappeared. For security, it means an accountable person accepted a documented residual risk. For cost, it means another manager can change an assumption and reproduce the result.

Pay attention to rejected options. A candidate who considered three interventions and explains why two were poor fits gives you more evidence than one who presents a single answer as inevitable. The rejection should cite your constraint: missing test coverage, sensitive data, vendor dependency, migration effort, or limited management capacity.

Require an evidence log, not a diary. Each entry should connect an observation to a decision:

Observation: 11 of 15 releases waited for the same reviewer.
Decision: test automated checks for routine migrations.
Unknown: whether the reviewer catches issues absent from current checks.
Test: replay checks and reviewer comments on the same 15 releases.
Result: [filled during trial]
Consequence: [continue, revise, or stop]

This log reveals whether the candidate updates beliefs. If new evidence contradicts the first diagnosis and the plan never changes, you are watching attachment to an idea. AI transformation creates many such moments because generated output looks convincing before it meets the repository, customer, or control environment.

The definition of done also protects your team from a trial that expands each day. A candidate may discover five adjacent problems. Record them, but do not add them to the assignment unless one blocks the agreed test. Scope control is part of the assessment.

Ask the candidate to state what remains manual. Mature leaders do this without embarrassment. They know automation should earn authority through observed performance, and that a small manual approval can be cheaper than building a control for a rare, severe action.

Force one security decision with consequences

The security thread should require a go, no go, or bounded go decision about an actual AI use case. Reviewing a generic policy tests writing. Deciding whether an agent may read a private repository, call deployment tools, inspect customer records, or open pull requests tests whether the candidate can connect business pressure to technical controls.

Pick one decision adjacent to the delivery bottleneck. If the candidate wants an agent to review migrations, ask what repository content it can read, what commands it can run, where prompts and outputs go, and who approves a change. Provide your current data classification and incident path if they exist. Do not ask the candidate to invent company policy during a short trial.

NIST's AI Risk Management Framework separates mapping, measuring, and managing risk, with governance applied throughout. That distinction is useful here. A threat list maps risk; a replay test or access review measures part of it; a permission boundary and named owner manage it. Candidates who label the threat list a mitigation have not finished the decision.

Require a short decision record with these fields:

Use case:
Business owner:
Data the system may receive:
Tools and actions allowed:
Actions requiring human approval:
Logs retained and reviewer:
Failure and abuse cases tested:
Rollback or access revocation:
Residual risk:
Decision and accountable approver:

Good judgment appears in the boundary. Read access to a selected repository is different from permission to merge. Creating a pull request is different from running a production migration. A human approval click has little value if the reviewer cannot see the command, affected environment, diff, and relevant test result.

Test at least one failure path. Give the agent a prompt or repository instruction that conflicts with the assigned task, remove a required credential, or make a tool call fail halfway through. The point is not to trap the candidate. You want to see whether the design fails closed, produces enough evidence for investigation, and returns control to a named person.

Never grant broad production access merely to make the trial realistic. Use a sandbox, a read only replica, a limited test environment, or recorded inputs when possible. If the use case cannot be evaluated without material customer or production risk, the candidate should say that the trial design is unsafe and propose a lower authority test.

Security theater has recognizable signs: a long risk register with no owner, "human in the loop" without a defined approval view, secrets placed in prompts, shared credentials, logs nobody will review, and controls whose operating cost never enters the cost model. A leader who finds a serious exposure should pause the intervention. Shipping on schedule is not a passing result when the conditions have changed.

Make the cost model survive a skeptical CFO

The cost model should compare the current workflow with a credible changed workflow, including the labor and control work that AI adds. A model based only on subscription price and engineer salaries is sales arithmetic.

Start with observed units. For the selected bottleneck, count items per month, median or item level touch time, waiting time, rework, and incidents. Assign loaded labor costs only where people actually spend time. Waiting in a queue has business cost, but it is not eight hours of engineer labor per day.

Use a model that exposes assumptions:

monthly_current_cost =
  current_touch_hours * loaded_hourly_cost
  + rework_hours * loaded_hourly_cost
  + expected_failure_cost

monthly_changed_cost =
  retained_touch_hours * loaded_hourly_cost
  + review_hours * reviewer_hourly_cost
  + tool_fees
  + integration_and_maintenance
  + expected_failure_cost_after_change

monthly_net_savings = monthly_current_cost - monthly_changed_cost
payback_months = one_time_change_cost / monthly_net_savings

Expected failure cost needs restraint. Multiply a plausible incident cost by an observed or explicitly assumed frequency, and label weak inputs. Do not manufacture precision with four decimal places. If you have no credible frequency, show the failure as a separate downside scenario instead of hiding a guess inside the headline result.

Demand at least three scenarios, even though forced groups of three usually add little. Here they serve a specific decision: conservative, operating, and failure. The conservative case assumes lower adoption and more review. The operating case uses the candidate's expected steady state. The failure case includes reversal, integration write off, and the cost of cleaning up poor output.

The model must account for displaced work rather than treating saved hours as cash. If an engineer saves ten hours but remains fully employed, payroll does not fall. The company may ship more, avoid a hire, reduce contractor spend, or remove a role over time. Each is a different benefit with a different evidence burden. A candidate who converts every saved hour directly into payroll savings is overstating the case.

Ask who owns every recurring cost after the trial. Prompt upkeep, evaluations, access reviews, incident response, vendor review, and model changes consume real time. Some controls will remain manual. Include them. AI work often looks cheap because the proposal counts generation and ignores verification.

Tie the model to a management action. For example: proceed if the conservative case avoids a planned hire within two quarters while the security control fits within existing review capacity; stop if review takes more time than the queue currently loses. A spreadsheet without a decision threshold becomes a prop.

At oleg.is, the Team & AI Audit uses a fixed five business day engagement and a savings threshold because founders need a bounded economic decision before a wider transformation. Your paid trial can use the same discipline even if you hire someone else: fixed scope, explicit evidence, and a result that finance can challenge.

Watch operating behavior under friction

Leadership skill appears when evidence is incomplete, an engineer disagrees, or a tool fails. The candidate's workshop presence and presentation polish matter less than the decisions made during those moments.

Attend two working sessions rather than every meeting. In one, watch the candidate diagnose the bottleneck with the owner. In another, watch the candidate review the security and cost tradeoff with an executive. You are looking for different behavior in each room: technical curiosity with the operator, then a concise recommendation and explicit uncertainty with the decision maker.

Ask the team for narrow feedback after the candidate hands off:

• Did the candidate understand the work before proposing a change?
• Could you challenge an assumption without being dismissed?
• Do you know what changed and why?
• Can you operate or reverse the change without the candidate?
• Did the work create an unplanned burden?

Do not ask whether people liked the candidate. A pleasant candidate can leave a fragile system, and a direct candidate can improve a tense queue. Ask about observable conduct and the condition of the work.

Strong candidates reduce ambiguity in public and correct themselves without drama. They write decisions down, assign owners with consent, separate facts from assumptions, and make dissent visible. They do not use an AI generated analysis as authority. They check the underlying record and take responsibility for the recommendation.

Watch for rescue behavior. Some candidates personally fix every script, rewrite every prompt, and run every meeting. The trial may look productive, but you have learned that output depends on them. A transformation leader should leave the existing team more able to operate the system.

Also watch whether the candidate can say no to the founder. If you push for production access, a larger savings claim, or a public launch before controls are ready, the candidate should explain the consequence and offer a bounded alternative. Agreement is easy to perform during a paid audition. Judgment sometimes creates useful friction.

Keep authority narrow while the work stays real

A trial can use real work without granting the candidate executive or production authority. Give enough access to inspect evidence and run the agreed test, then require your accountable employees to approve irreversible actions.

Create a temporary role with an expiry date. Prefer named accounts over shared credentials. Limit repository, ticket, analytics, and infrastructure access to what the charter requires. If a candidate needs more, require a written request that names the task, data, action, duration, and approver.

The candidate should never make personnel changes during the trial. They can identify role confusion, missing ownership, or work that no longer justifies its cost. The founder remains responsible for employment decisions and for the effect of the trial on the team. This boundary matters because people may interpret the candidate's questions as a hidden layoff assessment.

Tell the team what the trial is testing. Explain the scope, dates, access, expected outputs, and who will decide what happens next. Do not pretend the work has no cost implications if the assignment includes a cost model. State that no personnel decision will be made by the candidate and give employees a private path to report a concern.

Protect candidate confidentiality as well. A paid trial may expose methods, templates, or code the candidate developed previously. Your agreement should define ownership of the trial deliverables and preexisting material. It should also cover confidential information, security obligations, payment, and termination. Use counsel for language that fits your jurisdiction; a blog template is not a contract.

If the candidate's intervention reaches production, use the same change process you require from the team. Record tests, approval, monitoring, and rollback. "Temporary trial" is not an exemption. A candidate who asks to bypass normal controls has supplied a useful negative signal.

The trial stays real because the evidence, constraint, and decision are real. It does not need unrestricted access. In fact, designing a useful test inside a permission boundary is part of the job.

Review the decision trail, not the final demo

The final review should replay the candidate's important decisions from evidence to consequence. A demonstration can show that something works once. The trail shows whether your team should trust the leader with a transformation that lasts months.

Schedule ninety minutes with the founder, the bottleneck owner, a security owner, and whoever owns the budget. Ask the candidate to submit the artifact pack beforehand: baseline, experiment record, change or prototype, security decision, cost model, evidence log, open risks, and handoff. Participants should inspect the material before the meeting.

Run the review in this order:

1. Ask what the candidate believed on day one and what changed.
2. Replay one accepted and one rejected delivery option.
3. Challenge the security boundary with a plausible failure.
4. Change one cost assumption and recalculate the decision.
5. Ask the future owner whether the handoff is operable.

Do not spend half the meeting watching generated code. Inspect a sample where it affects the decision, but keep the discussion on cause, control, economics, and ownership. A candidate can hire or supervise specialists for implementation. The leadership question is whether the candidate directs technical power toward a defensible business outcome.

Score independently before discussion. Then compare large differences. The security owner may see an unowned control while the founder sees speed; the engineer may know that the proposed maintenance estimate is impossible. Independent scores prevent the most senior voice from setting everyone's memory of the trial.

Distinguish a failed intervention from failed leadership. If the change did not improve the bottleneck, but the candidate formed a sound hypothesis, protected the system, measured the result, stopped promptly, and left a better diagnosis, the trial may support hiring. If the metric improved because the candidate bypassed review and concealed cleanup work, the apparent success should count against them.

Require the candidate to name the strongest reason not to continue the transformation. This is not a debate trick. It tests whether the person can see organizational capacity, data quality, control cost, or weak economics that conflict with their own commercial interest.

Make the hiring decision while evidence is fresh

Hire or extend the engagement only when the trial shows repeatable judgment, not because the candidate created momentum. Decide within two working days, using the scorecard and written comments from the people who will inherit the change.

A passing candidate usually leaves five things behind: a sharper definition of the bottleneck, a tested change or a well supported stop, a security decision with an owner, a cost model whose assumptions can be changed, and a team that can continue without the candidate. Missing one item may be acceptable if the evidence explains why. Missing ownership is not.

Reject candidates who hide uncertainty, ask for broad access before defining the task, count generated output as delivery, turn saved time directly into payroll savings, or treat engineers as resistance to overcome. These habits become more expensive when the transformation expands.

If the evidence is mixed, do not blur the decision into an open ended advisory arrangement. You can commission a second bounded milestone aimed at the unresolved issue, with new acceptance criteria and payment. Do that only when the first trial discovered a question that could not reasonably have been known at the start.

Separate the role decision from the transformation decision. You may find a strong leader and a poor AI use case. You may also find a profitable use case and the wrong leader. The scorecard should let you say yes to one and no to the other.

A fractional CTO arrangement can be sensible when the work needs ongoing executive authority but not a full time hire. Founder advisory fits a different gap: it helps the founder decide, but it should not quietly inherit operational accountability. Name the role you are buying and match its authority to the result you expect.

The first month after a successful trial should preserve the same discipline. Keep the decision log, cost owner, access boundary, and rollback condition. Increase the agent's authority only after observed performance supports it. A leader who passed the trial will welcome that constraint because it turns an impressive week into an operating system the company can trust.