Source: [Workplace AI adoption: measure completed work, not logins — Skills for Business]() [← All articles](https://skillsforbusiness.ai/blog/) Workplace adoption · Guide # Workplace AI adoption: measure completed work, not logins A practical way to evaluate workplace AI adoption: choose a bounded task, count review and rework, and separate employee confidence from measured results. [Skills for Business](https://skillsforbusiness.ai/blog/editorial/)30 October 20267 min read On this page 1. [Research gives you questions, not a universal return](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#research-gives-you-questions-not-a-universal-return) 2. [Choose a task with a visible finish line](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#choose-a-task-with-a-visible-finish-line) 3. [Count the whole cost of getting to accepted work](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#count-the-whole-cost-of-getting-to-accepted-work) 4. [Compare like with like, and keep the exceptions](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#compare-like-with-like-and-keep-the-exceptions) 5. [Give HR, managers and developers different responsibilities](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#give-hr-managers-and-developers-different-responsibilities) 6. [Decide what to change before scaling](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#decide-what-to-change-before-scaling) 7. [Sources](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/#sources) What to take away - Measure the time and quality of an accepted outcome, including review and corrections. - Compare people doing similar tasks; a company-wide average can hide who benefits. - Treat a small pilot as a decision aid, not proof of organization-wide productivity gains. An AI rollout should earn its place by improving work that someone accepts and uses. The number of activated accounts tells you whether people have access. It does not tell you whether a customer gets a better answer, a new employee receives clearer onboarding, or a developer finishes a reliable change sooner. For a first pilot, choose one recurring task and measure it from request to accepted output. Include the time spent checking the AI’s work. Keep a separate record of quality failures. This is a proposed evaluation method, not a claim that every task can be reduced to a stopwatch. The distinction matters for organizations of every size. A small business can lose its expected savings to an owner’s evening corrections. A large enterprise can report strong adoption while quietly moving work from authors to reviewers. Both may look successful if the dashboard ends at “draft generated.” ## Research gives you questions, not a universal return The [2025 journal article Generative AI at Work](https://academic.oup.com/qje/article/140/2/889/7990658) studies 5,172 customer-support agents and reports a 15% average productivity increase, measured as issues resolved per hour. Gains varied across workers, with less experienced and lower-skilled workers benefiting more. That is evidence about a particular implementation and workforce. It is not a forecast for HR, engineering, or your own support team. We use the journal version here, rather than the earlier working-paper estimate of 14%. A different setting produced a different result. [METR’s July 2025 study](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) involved 16 experienced open-source developers and 246 tasks in familiar repositories. With early-2025 tools available, completion took 19% longer. Participants nevertheless believed AI had helped them. The result illustrates why perceived speed and measured completion time deserve separate treatment; it does not establish that AI generally slows developers. The later evidence also matters. In its [February 2026 update](https://metr.org/blog/2026-02-24-uplift-update/), METR said its newer experiment gave an unreliable signal of current productivity effects. Selection effects and difficulties measuring concurrent agent work complicated interpretation. Researchers considered greater speedups likely, but described their data as weak evidence for the size of the change. METR also published a [May 2026 survey of 349 technical workers](https://metr.org/blog/2026-05-11-ai-usage-survey/). Participants reported substantial gains, but the authors cautioned about self-reporting and selection. It adds a useful perspective on perceived value; it does not replace a measured comparison or validate a particular new model. Read these sources together and the useful management question becomes narrower: which tasks, for which people, under which conditions? Neither an old positive result nor an old negative result can settle that question for a current rollout. ## Choose a task with a visible finish line “Improve HR with AI” is too broad to test. “Prepare an onboarding checklist from the approved role description and current onboarding policy” is specific enough to inspect. The finish line is a checklist accepted by the person responsible for onboarding, with required steps present and invented requirements absent. Start with a task that occurs often enough to compare several examples. It should have identifiable inputs, an accountable reviewer and a tolerable failure mode. Drafting a routine internal document is easier to evaluate safely than making a consequential decision about an individual employee. Write down the current process before adding AI. Who gathers information? Who produces the first draft? Who checks facts? Who resolves exceptions? If you skip this step, an apparent improvement may simply reflect giving the pilot team cleaner inputs or more attention than everyone else. A useful pilot brief fits on one page: the task, eligible inputs, prohibited data, expected output, quality criteria, responsible reviewer, and conditions for stopping. Keep those conditions visible to participants rather than treating them as a hidden test. ## Count the whole cost of getting to accepted work For a routine document, record preparation, generation, review, correction and handoff time. Keep elapsed delivery time separate from active human time. An agent can run for ten minutes while someone completes another task; adding both clocks together would misrepresent the labor involved. Consider this hypothetical comparison: | Step | Existing process | AI-assisted process | | --- | --: | --: | | Prepare inputs | 10 minutes | 15 minutes | | Produce draft | 35 minutes | 5 minutes | | Check and correct | 10 minutes | 25 minutes | | Total active human time | 55 minutes | 45 minutes | The draft looks seven times faster. The complete task saves ten minutes, about 18% of the original human time. Neither figure includes software costs, setup or training. These invented numbers demonstrate the accounting method; they are not a forecast or observed customer result. Now add quality. A shorter process that regularly omits a required onboarding step may be unacceptable. Record whether the output passed, needed material corrections, or was abandoned. Do not assign a reassuring average score that allows a serious error to disappear among good grammar and attractive formatting. ## Compare like with like, and keep the exceptions A practical small pilot can use a few dozen representative tasks, but that is a starting sample, not a statistical guarantee. Include straightforward cases and inconvenient ones: incomplete inputs, conflicting instructions, unfamiliar terminology and requests outside the workflow’s scope. Where feasible, assign comparable cases across AI-assisted and existing methods. Avoid testing the same person on an identical case twice and attributing the benefit of familiarity to AI. Record task difficulty and experience level so that one group’s unusually easy cases do not distort the result. Let participants report frustration and confidence, too. Those measures matter for adoption and training. Just keep them separate from completion and quality measures. A pleasant tool may still require too much checking. A useful tool may initially feel awkward because people are learning a different process. Include abandoned attempts in the record. If someone spends twenty minutes with an assistant and then starts again manually, the attempt is part of the cost. Excluding it would make the tool appear more dependable than it is. ## Give HR, managers and developers different responsibilities HR can help define the learning objective, collect feedback without turning the pilot into employee surveillance, and identify where the process changes roles or support needs. The purpose should be to improve the workflow, not rank people by prompt counts. Managers should define an acceptable result and make time for review. If staff are expected to experiment on top of an unchanged workload, a pilot may measure their capacity for unpaid troubleshooting rather than the technology’s usefulness. Developers or technical owners should record the assistant product, model where visible, tool access and instruction version. They should also track failures that cannot be solved with better wording: missing permissions, stale documents or unavailable integrations. A [reusable agent skill](https://skillsforbusiness.ai/blog/skill-md-files-as-team-work-instructions/) can make instructions repeatable, but it cannot grant missing access. For a small business, one person may hold all three roles. The responsibilities still need to be explicit. For an enterprise, a lightweight shared pilot template can make results comparable without forcing every department into the same use case. ## Decide what to change before scaling At the review meeting, choose among four actions: continue the pilot, revise the workflow, expand to a closely related task, or stop. Explain the decision using accepted outcomes, review effort and failures. “People used it frequently” is useful context, not the conclusion. For example, a team might agree in advance to expand only if the pilot has no serious factual failures, meets the existing quality bar, and reduces median total human time by at least 10% across comparable cases. That is an illustrative decision rule, not a research-backed universal threshold. A small sample still warrants a limited next step rather than a company-wide conclusion. If most lost time comes from missing inputs, improve the intake form. If reviewers repeatedly correct the same defect, update the instructions and test a fresh set of cases. If the task requires judgment the system cannot reliably support, narrow the task or keep the existing process. Keep the successful workflow and its evaluation cases together. When a [new model arrives](https://skillsforbusiness.ai/blog/frontier-model-upgrades-business-evaluation/), rerun those cases before assuming the previous result still holds. The next useful milestone is a repeatable improvement in accepted work, with enough evidence to explain why it happened. ## Sources & further reading Sources checked for this article. Research findings and vendor claims are distinguished in the text. 1. [Generative AI at Work — Quarterly Journal of Economics](https://academic.oup.com/qje/article/140/2/889/7990658)Quarterly Journal of Economics; Brynjolfsson, Li and Raymond · Accessed 30 October 2026 2. [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)METR · Accessed 30 October 2026 3. [We are Changing our Developer Productivity Experiment Design](https://metr.org/blog/2026-02-24-uplift-update/)METR · Accessed 30 October 2026 4. [Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity](https://metr.org/blog/2026-05-11-ai-usage-survey/)METR · Accessed 30 October 2026 Produced with AI-assisted research and writing. Examples and recommendations are our editorial synthesis unless attributed. This article does not claim independent product testing or human expert review. [Read our editorial policy.](https://skillsforbusiness.ai/blog/editorial/) ## Keep reading [A new frontier model is a reason to test, not a reason to migrate](https://skillsforbusiness.ai/blog/frontier-model-upgrades-business-evaluation/)[SKILL.md files: turn team know-how into reusable AI instructions](https://skillsforbusiness.ai/blog/skill-md-files-as-team-work-instructions/)