Validation

Uncertainty bands

r.confidence_band is the single most load-bearing field in a claimcheck result, and the one most likely to get dropped when someone repeats a claim secondhand. This page covers how it’s computed and what it means when it’s wide.

How the band is computed

claimcheck does not average the confidence intervals reported by individual sources — those intervals are often computed under different methodologies and aren’t directly poolable. Instead, it treats the evidence set itself as the object of uncertainty: each source contributes a point estimate and a contribution weight, and the band comes from resampling across sources (weighted, with replacement) rather than within any single source’s data.

For the task-exposure claim used elsewhere in this documentation, the default procedure draws 2,000 resamples of the evidence set, recomputes a pooled estimate on each resample, and reports the 5th and 95th percentile of the resulting distribution — a 90% interval by default. That’s how (0.09, 0.19) around a 14% point estimate is produced: not as a hand-picked margin, but as the empirical spread you get from repeatedly asking “what would the pooled estimate look like if the evidence set had come out slightly differently.”

from equilibrium import claimcheck

r = claimcheck.validate("AI raised task exposure 14% in 2025", interval=0.90)
r.confidence_band  # (0.09, 0.19)

The interval argument controls the percentile width; narrowing it to 0.50 gives a tighter, less conservative band from the same resample draws.

What width signals

A narrow band means the sources in the evidence set agree closely and the pooled estimate is not sensitive to which subset of sources you happen to weight more heavily. A wide band means the opposite: sources disagree, the evidence set is thin, or a small number of sources are doing most of the work and the estimate would move substantially if one of them were removed or revised. Width is informative on its own — two claims with the same point estimate but different band widths are not equally well-supported, and shouldn’t be reported as if they were.

Reporting a wide band responsibly

Report the band, not just the point estimate — “task exposure rose 14%, with estimates from the underlying evidence ranging from 9% to 19%” is a materially different claim than “task exposure rose 14%.” If space is tight, report the point estimate with the band in parentheses rather than dropping the band entirely. Treat a wide band as a reason to look at r.evidence before repeating the claim, not as noise to average away — a band that wide is often telling you the underlying research hasn’t converged yet, which is itself worth saying.