Source: [AI-assisted business forecasts: make predictions resolvable — Skills for Business](<https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/>)

[← All articles](https://skillsforbusiness.ai/blog/)

Workplace adoption · Guide

# AI-assisted business forecasts: make predictions resolvable

Before using an AI forecast in business planning, define the event, evidence cutoff, baseline and outcome rules so its accuracy can be checked later.

[Skills for Business](https://skillsforbusiness.ai/blog/editorial/)1 November 20267 min read

On this page

1.  [What forecasting research can establish](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#what-forecasting-research-can-establish)
2.  [Replace “success” with a question someone can settle](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#replace-success-with-a-question-someone-can-settle)
3.  [Keep a forecast record beside the probability](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#keep-a-forecast-record-beside-the-probability)
4.  [Score a series, not a success story](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#score-a-series-not-a-success-story)
5.  [Preserve updates without rewriting the past](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#preserve-updates-without-rewriting-the-past)
6.  [Pilot the workflow before relying on the number](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#pilot-the-workflow-before-relying-on-the-number)
7.  [Sources](https://skillsforbusiness.ai/blog/ai-business-forecasts-resolvable-predictions/#sources)

What to take away

-   Define an observable event, deadline and resolution source before asking an assistant for a probability.
-   Preserve each dated forecast and compare it with a baseline made on the same information.
-   Evaluate a series of resolved forecasts; one successful prediction cannot establish calibration or business value.

Before using an AI-generated forecast in business planning, write down exactly what event is being predicted, when it must happen and which evidence will settle the result. Preserve the probability and information available at the time. Compare it with a simple baseline, then review a series of outcomes rather than celebrating one correct call.

A forecast such as “the rollout will probably succeed” leaves too much room for reinterpretation. A polished explanation can make that ambiguity harder to notice. The team may later remember a cautious statement as confident, change what success meant or credit the assistant for an outcome it never actually predicted.

Our editorial recommendation is to use a short forecast record before relying on the number. This article develops a hypothetical operational example and a proposed evaluation process. It does not report an assistant trial, customer result or independently tested forecasting product.

## What forecasting research can establish

[ForecastBench, published at ICLR 2025](https://proceedings.iclr.cc/paper_files/paper/2025/hash/ea74e45a229dac70b5b63b28d8934db6-Abstract-Conference.html), evaluates predictions about events whose outcomes are unknown when forecasts are submitted. Its questions come from forecasting platforms and real-world datasets. That design separates prospective prediction from answering a historical question whose result may already be in a model’s training material.

The original paper’s model comparisons describe its tested systems and conditions. They are not an estimate of every assistant available today. The [current documentation](https://forecastbench.org/docs/) distinguishes the conference paper from later changes, including a difficulty-adjusted scoring methodology for comparisons involving different question sets. An old score or an isolated leaderboard rank therefore needs its evaluation context.

There is also a difference between a benchmark question and a company’s planning problem. In a [practitioner critique of ForecastBench](https://goodjudgment.com/what-the-forecastbench-rematch-still-wont-measure/), Good Judgment questions how well the benchmark captures updating, collaboration and client question design. Good Judgment sells forecasting services, so this is an interested practitioner’s argument, not neutral validation of its own superiority. The useful challenge for a business reader is narrower: does the evaluation resemble the decision being delegated?

A public forecasting result cannot establish that an assistant knows your pilot customers, understands your operational definitions or uses the right internal evidence. Those are requirements for a local workflow.

## Replace “success” with a question someone can settle

Imagine a fictional business launching a supplier-ordering portal. Its operations manager asks an assistant whether the pilot will succeed. Before requesting a probability, the team proposes this question:

**Will at least eight of the ten companies on the fixed pilot list submit at least one valid order through the portal by 17:00 UTC on 15 December 2026?**

All companies, dates and figures in this example are hypothetical. A usable definition would also specify what counts as a valid order: for example, an accepted order in the production order log, excluding training transactions, duplicates and orders entered by internal staff on a supplier’s behalf.

That definition changes the assignment. The assistant is estimating one observable adoption event. It is not predicting profitability, customer satisfaction or the success of the entire business case. If the manager actually needs those answers, create separate questions with appropriate measures.

Agree on awkward cases in advance. If a pilot company withdraws, does it remain in the denominator? If an order arrives before the deadline but is validated the next day, which timestamp controls? If the log is unavailable, is the result pending or unresolvable? Decide these rules before seeing whether they help the forecast look good.

## Keep a forecast record beside the probability

For the fictional pilot, a compact record could contain the following fields. This is an editorial template, not a standard mandated by the research.

| Field | What the pilot team records |
| --- | --- |
| Event and deadline | Eight of ten fixed pilot companies place a valid order by the stated UTC deadline. |
| Resolution evidence | Approved order-log query, validity rules and an accountable outcome owner. |
| Information cutoff | Timestamp and versions of the readiness report, pilot list and available activity data. |
| Forecast | Probability, submission time, assistant/model identifier and tool access used. |
| Baseline | A separately recorded estimate from the existing planning method using the same cutoff. |
| Revision history | Every later probability, its timestamp and the new evidence motivating it. |
| Outcome | Yes, no, pending or unresolvable, with supporting evidence and any adjudication note. |

The record should make missing information visible. If the assistant cannot access the current readiness report, it should say so. A probability based on a generic description is a different submission from one based on verified pilot activity.

Keep source facts separate from assumptions in the rationale. “Six suppliers completed training” should point to an inspected report. “Training completion makes ordering more likely” is a judgment about that evidence. A useful rationale identifies what would change the estimate rather than merely finding reasons to support the number already given.

Use approved tools and proportionate records. The evaluation does not require copying confidential supplier correspondence into a broadly shared spreadsheet.

## Score a series, not a success story

Suppose the assistant records a hypothetical 65% probability and the existing planning method records 55%. If the event happens, both assigned it more probability than the alternative. Calling one “correct” and the other “wrong” would discard useful information.

One simple binary scoring convention is the squared difference between the forecast probability and the outcome, with the outcome coded as one for yes and zero for no. This is the binary Brier score used in the [ForecastBench paper](https://proceedings.iclr.cc/paper_files/paper/2025/hash/ea74e45a229dac70b5b63b28d8934db6-Abstract-Conference.html). Lower is better. In our hypothetical example, a yes outcome gives scores of **0.1225** for 65% and **0.2025** for 55%. A no outcome instead gives **0.4225** and **0.3025**.

These are illustrative calculations, not observed performance. One outcome cannot tell you whether the assistant’s probabilities are consistently well matched to reality. Compare the methods on the same collection of questions, with the same deadlines and information cutoffs. Keep unresolved and unresolvable questions visible rather than quietly dropping inconvenient cases.

Document the scoring convention. [Good Judgment Open’s FAQ](https://www.gjopen.com/faq) describes a Brier calculation that sums squared errors across answer options, giving twice the single-term score for a binary question. It also describes averaging scores over time. Those numbers should not be mixed directly with a single dated forecast scored under the convention above.

## Preserve updates without rewriting the past

A team can learn that two suppliers have postponed onboarding. Updating the probability may be appropriate. Replacing the old cell without retaining its timestamp destroys the evidence needed to evaluate the earlier decision.

Decide whether the comparison concerns the first forecast, the last forecast before a planning meeting or a series of updates. Apply that rule to both the assistant and the baseline. A forecast revised after decisive evidence arrives should not compete with an older estimate frozen before that evidence existed.

Also record interventions. If the forecast prompts the manager to provide extra onboarding support, the eventual outcome reflects that changed process. That may be valuable, but it complicates any claim that the original forecast alone demonstrated accuracy. A useful decision aid and an accurate passive predictor are related goals, not identical measurements.

## Pilot the workflow before relying on the number

Begin with a recurring, bounded operational question whose outcome becomes observable reasonably soon. Keep the existing planning method as a comparison and have an accountable person review the question’s definition. Neither a small business nor an enterprise needs a custom forecasting platform to preserve a dated record.

As results accumulate, inspect both average scores and the kinds of errors involved. Are high probabilities repeatedly attached to events that fail? Are supplier adoption questions weaker than delivery questions? A small or highly related set of outcomes supports only cautious conclusions; ten questions about the same rollout do not provide ten independent tests of general forecasting ability.

Keep decision usefulness separate from score improvement. A slightly better score may not change a staffing or rollout decision, while a useful warning may justify a modest preventive action. Define what the planning team will do with the estimate before treating an extra decimal place as valuable.

This discipline complements [evaluating AI adoption through completed work](https://skillsforbusiness.ai/blog/workplace-ai-adoption-measure-completed-work/). For forecasting, the completed work includes a question that can be settled, a preserved estimate and an honest outcome review. Start there before asking an assistant to sound more certain about the future.

## Sources & further reading

Sources checked for this article. Research findings and vendor claims are distinguished in the text.

1.  [ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities](https://proceedings.iclr.cc/paper_files/paper/2025/hash/ea74e45a229dac70b5b63b28d8934db6-Abstract-Conference.html)Ezra Karger and colleagues; ICLR 2025 · Accessed 1 November 2026
2.  [ForecastBench documentation](https://forecastbench.org/docs/)Forecasting Research Institute · Accessed 1 November 2026
3.  [Good Judgment Open FAQ](https://www.gjopen.com/faq)Good Judgment Open · Accessed 1 November 2026
4.  [What the ForecastBench Rematch Still Won't Measure](https://goodjudgment.com/what-the-forecastbench-rematch-still-wont-measure/)Good Judgment · Accessed 1 November 2026

Produced with AI-assisted research and writing. Examples and recommendations are our editorial synthesis unless attributed. This article does not claim independent product testing or human expert review. [Read our editorial policy.](https://skillsforbusiness.ai/blog/editorial/)

## Keep reading

[AI-generated customer interviews: separate hypotheses from findings](https://skillsforbusiness.ai/blog/ai-generated-customer-interviews-hypotheses-findings/)[AI-generated charts: define the comparison before the design](https://skillsforbusiness.ai/blog/ai-generated-charts-define-the-comparison/)