Before crediting an AI tool with a rollout’s entire improvement, identify what else changed. Staff may have received training, better source material, a clearer intake form and extra reviewer support at the same time. The resulting workflow may be valuable. That does not tell you how much value the assistant adds to the improved process.
Choose the comparison around the next decision. Continuing a combined service redesign is one decision; renewing an AI subscription within that redesign is another. This article proposes a comparison record and a fictional spending example to keep those questions separate. It is not a measured estimate of AI productivity or a ready-made statistical study design.
A current research result makes the distinction visible
In its 5 November 2026 assessment of Anthropic, METR separates observed changes in capability growth from estimates of what growth would have been without AI. It judged AI likely responsible for a majority of the current acceleration, with substantial uncertainty. That attribution is a judgment from several lines of evidence, not a randomized treatment effect. Anthropic redacted much of the evidence that METR found most informative.
The assessment concerns AI research at a frontier laboratory. It does not establish a productivity multiplier for an ordinary business. Our narrower inference is that an observed improvement and an estimate of AI’s contribution deserve different labels. We cannot independently inspect the redacted evidence or reproduce METR’s assessment.
Write the decision before choosing the baseline
Imagine a business improving its monthly supplier-report process. Previously, staff gathered inconsistent spreadsheets, wrote summaries and corrected them through several rounds. The new rollout introduces a standard intake form, source-checking training and an AI drafting assistant.
The finance owner wants to know whether the new service is worth continuing. The software owner wants to know whether to renew the assistant. They can share evidence, but they are asking different questions.
For the service decision, the relevant alternative might be the previous process. For renewal, the relevant alternative might be the improved intake and training with a manual drafting step. Comparing only against the old process can hide a less costly option that the business could now operate.
Write both alternatives in operational terms. State the inputs, instructions, training, support, review requirements and tools available in each. “Without AI” is incomplete if it could mean either the poorly organized old process or a substantially improved manual one.
Our completed-work guide explains how to define the accepted output and count review effort. Here the additional question is which feasible alternative the spending claim is comparing against.
One fictional record supports two different calculations
Suppose the business could obtain the following comparable results. These figures are entirely invented for arithmetic illustration: they are not observations, forecasts or evidence that the alternatives would perform this way. Each row covers the same monthly volume, quality requirement and review boundary.
| Operating arrangement | Human effort per month |
|---|---|
| Previous intake and manual drafting | 100 hours |
| Improved intake and training, manual drafting | 70 hours |
| Improved intake and training, AI drafting | 60 hours |
The combined rollout uses 40 fewer hours than the old arrangement: 100 − 60 = 40. The AI-assisted arrangement uses ten fewer hours than the improved manual alternative: 70 − 60 = 10. These are different contrasts. Neither calculation should silently borrow the other’s baseline.
For an illustrative financial screen, assume an agreed planning value of €40 per recovered hour. The combined difference corresponds to €1,600 a month before costs. The difference relevant to adding AI to the improved process corresponds to €400 a month before costs. If the assistant costs an invented €600 a month, that second comparison leaves a €200 shortfall on this narrow labor-value measure.
This does not mean the real assistant should be canceled. Other outcomes could matter, such as meeting a deadline or producing an additional useful analysis. It means the fictional €1,600 figure would not, by itself, justify the subscription against the stated alternative. Include any additional benefit explicitly and provide evidence for it.
Recovered hours also do not automatically become cash savings. The business needs a feasible use for them, and setup or training costs may require separate treatment. Our capacity allocation guide covers that next operational decision. The arithmetic here only makes the choice of comparator visible.
Do not fill an unobserved row with a confident guess
In practice, the middle row may never have been tried. The company might have moved directly from the old process to the combined rollout. In that case, do not present 70 hours as a measurement because a manager or assistant finds it plausible.
Keep estimates labeled. An assumption can support a sensitivity analysis: ask how the renewal decision changes across a range of possible manual-process costs. It cannot supply evidence that AI caused the difference. If the decision depends on that missing comparison, obtain better evidence before making a strong attribution claim.
Even observed rows need a defensible comparison. Different months may have different demand, staff experience or deadlines. A simultaneous comparison can also be misleading if managers assign the easiest work to one arrangement. Similar-looking totals do not resolve those problems.
For an evaluation, agree eligibility, outcomes and allocation before seeing the results. Where practical and appropriate, random assignment can help create comparable conditions. Team-level changes and shared training may require allocation at a different level from individual cases. Get suitable evaluation expertise when the decision needs a causal estimate; the three-row worksheet is not a substitute.
Published workplace research does more than subtract two averages
The 2025 journal paper Generative AI at Work provides a concrete methodological reference. Brynjolfsson, Li and Raymond studied a staggered assistant rollout among 5,172 customer-support agents. Section IV describes access following a three-hour onboarding session, with timing constrained by training capacity and deployment budgets. Its main rollout analysis uses difference-in-differences methods, including controls for calendar time, worker differences and tenure; the authors discuss assumptions and alternative estimators.
That is observational rollout evidence, not a simple randomized comparison of AI alone. Nor does it isolate the value of every implementation component. We cite the methods to show why a before-and-after subtraction needs more support, not to import its effect estimate into the supplier-report example. The study concerns a particular earlier system and workplace.
Training that enables people to use an assistant may properly belong inside the intervention being assessed. The aim is not to strip away everything that makes AI useful. Specify what a real rollout includes, then keep the claim at that level. “This supported workflow improved the outcome” can be the right question without becoming “the model alone produced the improvement.”
Preserve complementary changes instead of assigning every hour
The components may work together. Cleaner inputs might make the assistant useful; training might improve both manual and assisted work. Without suitable additional comparisons, the difference between two arrangements does not tell you how to distribute credit among all components.
Avoid allocating percentages of the gain to training, the model and data cleanup merely so they add to 100%. If the evidence supports only the package-level comparison, report that boundary. You can still retain a useful package while investigating a narrower purchasing question later.
Practitioner Martin Fowler’s June 2026 discussion of AI productivity metrics also treats measurement as difficult while retaining a role for qualitative feedback. That is professional commentary, not a causal study. Staff accounts can explain where the workflow helps or frustrates them; keep those accounts alongside the outcome comparison rather than treating them as a numerical attribution method.
Attach the alternative to the approval record
Our proposed decision record names the action, the feasible alternative, what differs between them, the evidence available and the unresolved assumptions. Include the quality boundary and the costs that actually change with the choice. Keep historical setup costs separate from avoidable future costs when reviewing a renewal.
For the fictional business, the record might say: “The combined arrangement is compared with the old process. Renewal requires a separate comparison with improved manual drafting. That alternative has not yet been measured, so the current record does not establish the assistant’s incremental labor benefit.”
A small business can use this statement to decide whether a bounded comparison is worth the effort. An enterprise can use it to prevent different departments from crediting the same process improvement to several separate purchases. Before the next approval, make the alternative explicit—and keep unknown results unknown.