Skip to main content
Methodology

Measurement confidence

Measurement confidence combines coverage, source fidelity, repeatability, freshness, and validation. These are technical indicators, not guarantees that an answer is true.

View confidence dimensions See how missing data is handled
Coverage of stored data Answer repeatability Input data quality
Visualization for page: Measurement confidence
Definition and protocol

Coverage of stored data

Coverage is valid observations divided by planned observations. Separately, we show k/n and a 95% Wilson interval for the mention rate.

A production run with no valid observations fails closed and does not publish a report.

Methodology rule

Answer repeatability

The same question asked of an AI model several times can produce different answers. That is why we repeat queries and measure result stability — the more the answers agree, the higher the measurement confidence. High answer variance lowers confidence and is reported openly as a property of the environment, not a measurement error.

Methodology rule

Input data quality

The pipeline separates valid answers, failures, and unavailable AI platforms before scoring. It stores statuses, provider errors, platform context, and performs automated report QA.

A partial but non-zero measurement may be shown only with explicit coverage and interval width.

Methodology rule

Run and history freshness

The current run has a timestamp, and audit history contributes to the change component. Missing history lowers a supporting freshness dimension.

Older individual answers do not currently receive a separate age-decay score. Measurement date must be considered during interpretation.

Methodology rule

Component directness

Components receive a real, proxy, or declared source status. Source fidelity also contributes to overall run confidence.

The status is not yet a complete metric for every individual answer. In particular, a model name does not prove its consumer application was measured.

Methodology rule

Confidence bands

Instead of a deceptively precise number, we communicate confidence in readable bands — from a complete measurement to a significantly limited one. The band immediately tells you how much weight the score and its changes can bear. The criteria for band assignment are published in the methodology.

Methodology rule

Partial measurement

A partial run can result from a missing provider, limit, refusal, API failure, or too few repetitions. Each such item keeps its error code and is not counted as “absent”.

A real “absent” value comes only from a successfully completed answer in which the brand was not detected.

Methodology rule

Effect on interpretation

Confidence changes how a score should be read: with high confidence a shift of a few points can be a signal, with low confidence it sits within measurement noise. Reports pair every score delta with its confidence band to prevent over-interpretation. It is a safeguard against conclusions the data does not support.

Continue

Related GeoFolks areas

Compare plans
Methodology A traceable GEO methodology

The headline result is the observed mention rate on a frozen panel. We show k/n, a 95% CI, coverage, AI platform context, and separate readiness diagnostics.
Methodology Providers and measurement channels

GeoFolks uses configured API integrations. A model name, consumer application, and API channel are not the same environment.
Methodology Measurement quality control

Automated checks run for every report. Human review is confirmed only by a recorded QA status and reviewer.
Next step

Check result reliability

Measurement confidence combines coverage, source fidelity, repeatability, freshness, and validation. These are technical indicators, not guarantees that an answer is true.

View confidence dimensions Discuss scope