Measurement failures that quietly mislead creative decisions, and how to check for each

The costly errors happen before a chart is drawn: an output is mistaken for an outcome, a conversion changes meaning, or a credited action is presented as causal proof. Correct the measurement design before buying another dashboard.

What to take away

How this list was selected

Inclusion required a failure that can change a creative or campaign decision and is addressed by a current authoritative record. Research date: 5 September 2026.

Geography is England for the reader's operating context; cited UK rules keep their stated remit, and global platform pages cover their own products. Exclusions were price comparisons, vendor rankings and alleged performance rates, and the seven entries are ordered by workflow, not severity.

1. Starting with available fields

A tool may expose renders, edits and exports, but none defines why the campaign exists. Write the intended audience change first, then pick the smallest set of measures to judge it.

GovS 011 Communications asks in-scope public bodies to focus objectives on outcomes and monitor outputs, outtakes and outcomes. Private organisations can borrow the distinction without claiming the standard applies to them.

2. Losing the creative specimen

Aggregated campaign rows cannot explain which approved message, image, format or version reached the placement. Preserve creative, job and release IDs through the media export. Reconcile them before interpreting differences. GDS guidance on using performance data distinguishes raw measurements from calculated metrics, a useful check when a dashboard mixes asset counts with rates.

3. Letting a conversion drift

A renamed event may keep the same column while its trigger, value or eligibility changes. Freeze an event card and mark revisions. Google Ads' conversion measurement page documents different action sources and the distinction between primary and secondary conversions. That product definition should be recorded when its data is used.

4. Treating credited actions as incremental

Attribution divides or assigns credit under a rule or model. It does not by itself reveal the counterfactual. Google's attribution model documentation states that the chosen setting affects conversion reporting and bidding. Call the number attributed conversions, then use a suitable experiment or evaluation if the decision requires an effect estimate.

5. Changing several causes at once

A new creative often launches with different targeting, budget, placement and offer. Any movement then has several possible explanations. The Magenta Book methods annex explains why experimental and quasi-experimental comparisons depend on design. Isolate the treatment where feasible and log deviations.

6. Ignoring collection permissions

Adding a pixel because reporting has a gap is not a privacy assessment. Identify the technology, purpose, data flow and user choice before implementation. The ICO's final storage and access guidance covers cookies, tracking pixels, tags and exceptions after its April 2026 update. A qualified adviser must apply it to the actual setup.

7. Hiding uncertainty in presentation

A precise total may still have incomplete coverage, late events, modelling, a changed denominator or an unstable comparator. Put those limits beside the figure and retain revision history. The Government Analysis Function's advice on quality and uncertainty says transparency helps prevent conclusions the evidence cannot sustain.

The corrective action

Select one live report and trace a number backwards through its definition, source extract, creative ID and collection authority. If the chain breaks, label the result provisional and assign the repair. If it holds, record the decision it informs and the boundary beyond which it should not be reused.

Before you act

Common questions

Why is starting with available fields a measurement failure?

A tool may expose renders, edits and exports, none of which defines why the campaign exists. Write the intended audience change first, then select the smallest set of measures needed to judge it. GovS 011 asks in-scope public bodies to focus objectives on outcomes and to monitor outputs, outtakes and outcomes.

What is the difference between attribution and incremental effect?

Attribution divides or assigns credit under a rule or model. It does not by itself reveal the counterfactual. Google's attribution model documentation states that the chosen setting affects conversion reporting and bidding. Call the number attributed conversions, then use a suitable experiment or evaluation if the decision requires an effect estimate.

How should uncertainty be handled in presentation?

A precise total may still have incomplete coverage, late events, modelling, a changed denominator or an unstable comparator. Put those limits beside the figure and retain revision history. The Government Analysis Function's advice on quality and uncertainty says transparency helps prevent conclusions the evidence cannot sustain.