How to review an outside CTO after 90 days without guesswork

Why this review goes wrong

Most founders judge an outside CTO by how the relationship feels. If meetings are smooth, replies come fast, and the person sounds confident, the work can look stronger than it is. That is a weak way to judge the first 90 days.

Charisma hides weak delivery more often than people admit. A polished CTO can calm the room, explain delays well, and make unfinished work sound temporary. Meanwhile, the product ships at the same slow pace, bugs stay open, and the team still lacks a clear plan.

Fast replies fool people too. Slack speed feels like involvement, but it does not prove good judgment. A CTO can answer every message in five minutes and still make bad calls on hiring, architecture, priorities, or cloud costs.

The first 90 days should leave visible changes. Releases should go out with less drama. Engineers should stop rebuilding the same feature twice because priorities are clearer. The roadmap should start matching what customers need instead of whoever shouted last in chat.

Spend matters just as much. If infrastructure is messy, a strong outside CTO should spot it early. You should see cleaner hosting choices, fewer unused services, or a simple plan to cut waste without hurting uptime.

A lot of reviews go wrong because founders ask, "Do I trust this person?" before they ask, "What changed?" Trust matters, but results come first. Good technical leadership shows up in the product, in the team, and in the numbers.

Picture a small startup after three months. The founder likes the CTO because he is always available and sounds sharp on every call. Yet deadlines still slip, developers still disagree on what to build next, and the cloud bill keeps climbing. That is not a strong result. It is a good impression.

If you want to review an outside CTO without guesswork, start with visible changes people can name and measure. Personality counts, but only after the work shows up where it should: product progress, team behavior, and spend control.

What changed in the product

After 90 days, the product should look different in ways a customer can notice. New screens are nice, but shipped work matters more than slide decks, roadmaps, and long architecture talks.

Start with one plain question: what reached users? Ask for a short list of releases, fixes, and changes that went live during the period. If the outside CTO mostly produced plans, meetings, and reorg ideas, your product probably did not move much.

Good product change often looks smaller than people expect. One of the best signs is fewer priorities, not more. If the roadmap got shorter, the team may finally know what matters now and what can wait.

That matters because weak CTO work often creates motion without progress. The backlog grows, the product story gets more complicated, and nobody can point to the few changes that should lift activation, retention, or reduce support load.

Use a few direct checks:

• Which items moved from backlog to real users?
• Which items got cut because they distracted the team?
• Which product problem got easier to explain after 90 days?
• Which delay still has no clear owner or reason?

That last question says a lot. Delays happen. A strong CTO can name the cause, the tradeoff, and the new date. A weak one keeps everything vague. If a feature slipped because data was messy, design changed, or the team lacked a skill, you should hear that clearly. If you still get foggy answers, the product work is not under control.

Say a startup began the quarter with 25 roadmap items. Ninety days later, a good fractional CTO review should show that three or four things shipped, one painful user flow got shorter, and several low impact ideas got removed. Maybe the team dropped a custom admin rebuild and fixed signup friction instead. That is real product movement.

When you review an outside CTO, look for fewer promises and more proof. Users should have something new to use, the team should have a simpler product focus, and any missed work should come with a direct explanation.

What changed in the team

You can learn a lot by watching how the team works on a normal Tuesday. After 90 days, roles should feel clearer. People should know who owns architecture, who makes the final call on tradeoffs, and who can approve day to day technical decisions without waiting for the founder.

If that still feels fuzzy, the outside CTO may be talking more than fixing. A good team does not need to ask the same person about every small choice. Engineers should know their area, product people should know where to bring requests, and urgent issues should not bounce through three chats before anyone acts.

Clear decision making matters more than new process docs. Ask the team one simple question: "Who decides what now?" If three people give three different answers, the structure is still weak.

Blocked developers are another honest signal. Strong CTO work usually cuts waiting time. Developers spend less time stuck on missing access, unclear priorities, slow code review, or surprise changes in direction.

Look for signs like these:

• Fewer "I am waiting on..." messages in Slack
• Faster answers on technical tradeoffs
• Less rework after a task is already built
• Clear owners for bugs, releases, and architecture

Meetings tell the truth too. The team should not need more calls to feel aligned. In many cases, a good outside CTO cuts meeting time by setting better ownership, cleaner specs, and tighter decision rules.

That does not mean every meeting should disappear. It means the useless ones should. A 45 minute status call that turns into a 15 minute planning check is real progress. So is a team that writes things down once instead of repeating the same discussion in three separate calls.

For example, before the new CTO joined, two developers might have waited a full day for approval on database changes while the founder sat in every planning meeting. After 90 days, one engineer owns that area, reviews happen the same afternoon, and the founder only joins when a decision affects money or roadmap.

That kind of change looks boring. Good. Team health usually looks boring in the best way: fewer stalls, fewer meetings, and less confusion about who does what.

What changed in spend and infrastructure

Money leaves a clear trail. If you want to review an outside CTO after 90 days, pull the bills from the three months before they started and compare them with the latest full month. Look at cloud hosting, databases, monitoring, CI/CD, third party APIs, and software licenses.

A lower bill is good only if the product still works well. If spend dropped by 30% but outages went up, pages got slower, or the team lost tools they used every day, that is not a win. Cost cuts count only when uptime, release speed, and user experience stay flat or improve.

Tool count matters too. Outside CTOs often inherit a messy stack: two error trackers, three project tools, duplicate cloud services, and old seats nobody uses. Cleaning that up is real work. It saves cash every month and makes the team faster because people stop jumping between overlapping tools.

Ask for direct answers:

• Which services got removed?
• Which ones got merged into one tool?
• Which servers, databases, or plans got sized correctly?
• Which licenses were canceled because nobody used them?
• What stayed in place because removing it would create risk?

The best answers sound boring and specific. "We moved from four small databases to one managed cluster" is useful. "We optimized infrastructure" tells you nothing.

Imagine a startup spent $11,000 a month across cloud hosting, logging, CI runners, and old SaaS tools. After 90 days, the bill drops to $7,800. That looks good, but you still need proof: uptime stayed steady, incident count did not rise, and the team still shipped on time. In that case, the savings are real.

This is where good CTO judgment shows up. Cutting spend by deleting things is easy. Cutting spend by removing duplicate services, shrinking oversized machines, and keeping the product stable takes skill. Ask for a simple before and after table with monthly cost, tool count, uptime, and incident volume. If they can show that clearly, you have something solid to judge.

How to run the 90-day review

To review an outside CTO fairly, set the review up on day one, not on day ninety. Write down the job in plain numbers: what should change in the product, what should get easier for the team, and what should cost less or run with fewer surprises.

A baseline can be simple: release frequency, bug backlog, cycle time, uptime, cloud bill, contractor spend, support load, and open hiring gaps.

By week twelve, collect proof from the same places. Pull release notes, tickets closed, bugs reopened, incident logs, cloud invoices, subscription costs, and any process documents the CTO introduced. Do not reward effort on its own. Check whether the work changed the business in ways other people can see.

Short interviews help. Speak to a founder, one product person, and two engineers separately. Ask the same questions each time: what got easier, what still feels messy, what change actually stuck, and what promise never landed. People usually speak more plainly when a manager is not in the room.

A fractional CTO review does not need a long slide deck. A simple scorecard works better because it forces clear judgment:

• 0 = no clear movement
• 1 = partial change, but still fragile
• 2 = real change the team can keep without constant rescue

Score each original goal against evidence, not memory. "Cloud spend down 22% after rightsizing" says more than "infra improved." "Two releases shipped without weekend fixes" says more than "delivery feels better." If the CTO claimed better hiring, ask whether interviews got faster, role definitions got clearer, or a bad process simply got renamed.

Keep the meeting short and finish with one decision. Renew with a fresh 90 day scope and numbers, or replace the CTO and write down why. Avoid the soft middle where everyone says the work was "good" but no one can point to product gains, team habits, or lower spend. If those three areas moved in the right direction, you likely have a solid 90 day CTO assessment. If they did not, extra time rarely fixes the problem on its own.

A simple example

A six person SaaS team brings in a part time CTO for two days a week. Before that, the founder approves too many small technical decisions, releases slip past the end of each month, and the team keeps paying for tools nobody really uses. Everyone is busy, but progress feels muddy.

Ninety days later, the picture is clearer. The CTO removes two paid tools that overlap with things the team already has and stops a side project that sounded good but never helped sales, retention, or support. That alone cuts waste and frees time the team had been splitting across too many directions.

The product changes are easy to see. Instead of one larger release that often misses its date, the team ships smaller updates every week. Bugs still happen, but they get fixed faster because the work is smaller and easier to test. Customers notice steady movement instead of long quiet periods.

The team changes are just as visible. Developers spend less time waiting for answers because the CTO sets clear rules on who decides what. A senior engineer can approve routine technical choices. The founder handles business tradeoffs. The CTO steps in for architecture, hiring gaps, and risky product calls. That removes a lot of stop and start work.

By day 90, the founder can point to a few plain results:

• 2 unused tools canceled
• 1 weak project stopped
• Releases moved from late monthly batches to weekly updates
• Fewer blocked tasks sitting in Slack waiting for approval
• More developer time spent building instead of debating

That is what a useful review of an outside CTO should look like in practice. You are not grading style, confidence, or how fast someone replies to messages. You are checking whether the product ships more often, whether the team waits less, and whether spend makes more sense.

If a part time CTO has been in the seat for three months and none of those things changed, that tells you something. Advice may have been given. Leadership probably was not.

Mistakes that hide weak work

After 90 days, the easiest trap is to mistake motion for progress. An outside CTO can look busy all day, answer Slack in two minutes, and still leave the product in the same place. Fast replies feel reassuring, but they matter only if they lead to better releases, fewer repeat problems, and clearer decisions.

Roadmaps hide weak work too. A long plan with neat phases and future milestones often buys more time than it should. When you review an outside CTO, shipped proof matters more than planned proof. Early work does not need to be flashy, but it should leave a mark: a painful bug fixed, a release process cleaned up, a blocked feature moved forward, or a messy backlog cut down to something the team can actually use.

Spend creates another blind spot. Teams often forgive rising costs because things feel calmer. Maybe the CTO added tools, more cloud capacity, or outside help, and daily work now feels less chaotic. Calm is nice. You still need to ask what that calmer setup produced. If monthly spend climbs and uptime, delivery speed, or product quality do not improve, the team bought comfort, not progress. Good engineering leadership often cuts waste before it asks for more budget.

The worst mistake is letting the CTO grade their own work. If the review depends on their slide deck, their wording, or their explanation for why results are "still coming," you do not have a review. You have a performance story.

Use evidence from four places instead:

• Shipped product changes
• Team output and ownership
• Infrastructure or cloud spend
• Unresolved risks that still block growth

Imagine a startup that now has better standups, more documents, and a much busier Slack workspace. The founder feels less alone, so the CTO seems effective. But after 90 days, release speed is flat, one senior engineer still handles every fire, cloud bills rose 30%, and the roadmap is still mostly promises. That is not a strong fractional CTO review. It is a smoother looking version of the same problem.

A good outside CTO should make their own score harder, not easier. They should bring numbers, show tradeoffs, and invite the founder or board to test the claims. If they cannot do that, do not let confidence fill the gap.

A quick checklist before you decide

Before you decide, cut the noise and look for proof. You should see changes in what users got, how the team works, and how money gets spent. If most of the evidence is meetings, opinions, or fast replies in chat, that is a weak result.

Answer these questions with examples, numbers, or both:

• Can you point to two or three things that shipped and made the product better for users?
• Did daily work get easier for the team?
• Did costs go down, or at least stay flat while output improved?
• Are the biggest risks smaller now, and are the rest named clearly?
• Would you hire this person again for the next phase?

One weak area does not always mean failure. Early work sometimes goes into cleanup, hiring, or infrastructure that users never see right away. Still, the outside CTO should be able to connect that work to a visible result, such as fewer incidents, faster releases, or lower cloud spend.

Be strict about evidence. "I liked working with them" is not enough. "They reduced outages, made deployments routine, and kept our costs under control" is enough. If your answers stay vague after 90 days, the role is probably too loose, or the person is not doing the job well enough.

What to do next

Your next move should come from evidence. If the last 90 days gave you clear product movement, better team habits, and lower waste in infrastructure, keep going. If you still cannot point to concrete changes without a long story attached, that is your answer too.

Write down the next 90 days in plain language. Skip vague goals like "improve engineering" or "modernize the stack." Put numbers and visible outcomes on paper so everyone can check them later.

A short plan usually works best:

• Ship a specific product milestone by a set date
• Cut open bugs or support issues by a clear amount
• Reduce cloud and tool costs to a target monthly number
• Tighten team process, such as faster code review or fewer blocked tasks

If the CTO made real progress and you can see the pattern holding week after week, keep them. Steady work matters more than a few heroic saves in Slack. A good outside CTO should make the product easier to ship, the team easier to manage, and the bills easier to explain.

If progress stays fuzzy, replace them. Do not give extra credit for confidence, fast replies, or polished status updates. If you cannot tell what changed in the codebase, roadmap, team routine, or cloud bill, you are paying for motion without proof.

Mixed results need a firm reset. Give one more 90 day block only if you can name the gaps and agree on exact targets now. Put review dates on the calendar. If those dates arrive and the same questions still hang in the air, move on.

Sometimes the hardest part is getting an unbiased read. If you need a second opinion, Oleg Sotnikov at oleg.is does this kind of fractional CTO advisory, with a strong focus on product architecture, infrastructure, and practical AI adoption. That kind of outside review can help you confirm whether the work is solid, stalled, or heading in the wrong direction.

The next decision does not need drama. It needs a written plan, a clear owner, and results you can see without guessing.