Measuring branding agency work across five evidence layers

These layers describe different strengths of evidence, not stages that every branding project must complete. A buyer or agency should select the layer that fits the decision, then label conclusions no more strongly than the available design allows. The distinction matters when measuring a branding engagement.

This material concerns buyers and agencies working with organisations in England. The cited Treasury and digital-service sources were prepared for government, so they offer analytical disciplines rather than private-sector requirements.

Frame the evaluation around a live choice

Begin with the decision the evidence must inform: acceptance of contracted work, correction of a rollout problem, continuation of a controlled trial, or commissioning a stronger evaluation.

Name the decision-maker and timing, and state what evidence would prompt a pause. This keeps collection connected to action rather than to a dashboard for its own sake.

The Magenta Book, written for government policy, separates process, impact and value-for-money questions. That distinction can help a private buyer avoid treating completed creative work as evidence of impact. The Green Book also connects appraisal with monitoring and evaluation for central government decisions, without imposing that governance framework on private agencies.

Keep a credible no-change or repair option visible when comparing interventions. A rebrand is not the only possible route, and excluding retention or repair of an existing identity can make the selected programme appear stronger than the decision evidence warrants. Set out the alternative before resources are committed and before judging later results.

Delivery proof: did the agreed work pass?

Operational evidence establishes whether specified work occurred. It may record an approved naming decision, an accepted asset, an authorised training session or a completed handover. Count only events that satisfy a documented state definition; a file being sent does not by itself mean that it has been accepted.

Use both the agency register and the client acceptance record, and retain rejected, withdrawn and corrected states. This supports delivery management, but it does not demonstrate audience response or commercial value. A disagreement between records is itself a reporting issue to resolve, not a reason to silently substitute one source for another.

Observation: what changed in the defined group?

Descriptive analysis reports what happened for a stated population and period. It might show the share of eligible assets passing a test or a shift in responses to a defined research instrument. Report the denominator with the result; a percentage without its eligible base leaves the scale and meaning unclear.

Observation does not explain the cause of a movement. If the pre-launch and post-launch populations, questions, recruitment routes or time windows differ, identify the break rather than presenting a continuous trend. Explain what was measured and where coverage changed before inviting readers to interpret a difference.

Attribution: where does the recorded credit go?

An agreed rule, such as first contact, last eligible contact or a weighted sequence, assigns recorded events for accounting or workflow; windows, deduplication, cross-channel exposure and unknown cases need definitions, and the assigned total is not proof of incremental effect.

See attribution methods and return on investment for the wider treatment.

Contribution: does the proposed route have support?

A contribution case examines whether evidence supports a plausible path from branding work to an observed outcome, while considering competing explanations. The Treasury analytical-method annex describes contribution analysis as building an evidenced line of reasoning, not as providing definitive proof of a causal effect.

Record the proposed theory, evidence that implementation occurred, expected intermediate changes, contrary evidence and plausible alternatives. State the judgement in bounded language. Saying that the evidence is consistent with a contribution communicates a different level of certainty from reporting a quantified effect.

Causal impact: what is the counterfactual?

Causal evaluation estimates the difference between an observed outcome and a credible account of what might have happened without the intervention. Randomised or suitable quasi-experimental designs can address that question when the design assumptions and delivery are defensible. A before-and-after movement alone does not supply the missing comparison.

Brand rollouts can make this work difficult because exposure may be widespread, timing may coincide with other changes and eligible comparison units may be scarce. Those limits call for a narrower claim, not a weaker label for the method. A qualified evaluator should choose and assure any design against its conditions.

Worked example: one rebrand across the five layers

The figures below are invented to show how each layer reads. A regional services firm commissions a rebrand from a branding agency.

Delivery: 14 of 16 contracted assets are accepted against their criteria, one is in correction and one is untested. That is a delivery record only.

Observation: in the three months after launch, 62 of 80 staff surveyed say they use the new templates. The denominator is the 80 staff surveyed, not all staff.

Attribution: the firm's website form has a "how did you hear of us" field, and 9 of 40 enquiries cite the new campaign. That is credit under a stated rule, not incremental enquiries.

Contribution: interviews and enquiry notes suggest a plausible route from clearer positioning to better-qualified enquiries, alongside a new sales hire as a rival explanation. The judgement is "consistent with a contribution".

Causal impact: with no comparison group, no causal estimate is claimed.

Using the layers to compare agencies

When you review branding agencies, ask each which layers it reports on, and in what words. An agency that reports delivery and observation clearly is easier to hold to account than one that quotes a causal effect without a comparison. Compare the answers alongside its review evidence pack and its contract terms.

Specify each measure before collecting it

A label such as engagement does not define a metric. Each measure needs a card identifying its purpose, decision, owner, measured event, eligible population, numerator, denominator, unit, period and cut-off. It should also set out inclusion and exclusion rules, source fields, missing records, uncertainty and the person responsible for assurance.

Also specify how late, missing, corrected and withdrawn records are handled, which sources of bias are known, and what threshold prompts review or a stop. This makes a review trigger a defined response rather than an improvised reaction to a surprising number.

A blank formula can expose questions before real values are inserted: accepted eligible outputs / all eligible outputs submitted for acceptance. The formula still needs state rules. Decide whether a corrected file counts as a version of one output or as a separate submission, then apply that decision consistently so comparisons remain interpretable.

Keep collection design apart from calculation

Map each field from its creation through storage, change and deletion. Identify who supplies it, what establishes the event, where it is held, who may amend it and how a correction travels. Separately document the calculation: eligible rows, joins, exclusions, cut-off and the form of the reported result.

This separation helps reveal missing events, duplicate identifiers and populations that changed after the baseline. Reconcile counts where records move between states before interpreting a result. The Aqua Book distinguishes verification, whether analysis follows its specification, from validation, whether it suits its intended use; a correct recalculation does not validate the business conclusion.

For repeatable reporting, retain the definition, query or code, input snapshot, transformations, reviewer and release record. The Government Analysis Function's reproducible analytical pipelines strategy describes software-engineering practices intended to improve reproducibility. A spreadsheet can apply the same discipline through controlled inputs, formulae, change logs and independent checks.

Protect people while retaining an audit route

Do not gather personal information simply because a reporting tool can display it. Data protection by design and default means considering protection from the outset and throughout processing.

The appropriate lawful basis, roles, transparency, access, retention and rights handling depend on the actual purpose and circumstances; obtain a data protection review of the real flow.

Separate a privacy-safe management view from controlled case-level evidence, using aggregation or suppression where disclosure risk warrants it and preserving an authorised reconciliation route. A privacy review cannot certify statistical validity, and statistical usefulness cannot override a failed privacy gate. Research participation, service administration and direct marketing remain distinct purposes.

Release results with their quality visible

Place the definition version, latest complete period, data freshness, missingness, revision status and known breaks beside the result. Keep preliminary and final releases distinct.

The UK Statistics Authority's Standards for Official Statistics call for openness about strengths, limitations and uncertainty; they are a demanding reference, not a claim that an agency report is an official statistic.

For sample-based research, explain the sampling frame, recruitment, responses achieved, weighting, non-response and uncertainty. The ONS explanation of survey uncertainty distinguishes sampling error from other problems, including inaccurate answers and processing mistakes. A confidence interval attached to a convenience sample must not suggest a representativeness that the sample does not have.

Qualitative evidence needs its own boundaries: eligibility, recruitment, questions, coding, disconfirming evidence and groups not represented. A quotation records what one participant said in context; it does not establish how common that view is. The Code of Practice for Statistics, the Government Data Quality Framework and the quality framework for government analytical outputs are further official reference points.

Different reporting rhythms serve different needs. An operations view can flag a broken feed or stalled approval; an evidence review can investigate exceptions and revisions; a decision pack can show the proposed theory, observations, alternatives and uncertainty. The ONS working papers provide another official route for methodological material; no single crowded screen needs to carry every purpose.

Treat comparisons as a specified research task

Before using a market benchmark, define the target population, geography, service category, cohort, period, measure, unit and collection method, then compare only compatible observations; if none qualifies, report that no suitable benchmark was found and use an internal baseline only as that organisation's history, not an industry norm.

See benchmark research.

HM Treasury's Test and Learn guide describes structured testing and adaptation as an approach rather than a type of evaluation. An iteration log can preserve what was learned during a branding programme without presenting that learning record as an impact study.

Make assurance and correction part of release

Before publication, one person should check the calculation and another should challenge whether it is fit for use.

Material outcome claims need approval from a named statistical or evaluation reviewer; privacy, finance and commercial reviewers should assess their own areas. Keep a release record of inputs, method version, checks, limitations and approval.

When correcting an error, state what changed, why, which periods are affected and whether earlier decisions need reconsideration. Retain a superseded version where lawful and proportionate so an authorised reviewer can reconstruct the decision. Name in advance who can pause reporting and what evidence permits it to restart.

Stop a measure if its eligible population cannot be recovered, a join loses material records, a definition changes without a bridge, privacy controls fail or the assumptions behind a causal design no longer hold. These are reasons to review the evidence route, not to conceal a break or relabel the remaining result.

A defensible conclusion uses language matched to the layer: delivery records show acceptance against recorded criteria; observation describes change in the specified group; attribution reports allocation under a rule; contribution offers a qualified judgement; and only adequately designed and assured impact evaluation should carry a causal estimate.