Measurement
Part of Privacy-first measurement: measurement register template
Four dashboard statuses, from usable to withheld
Give each measurement dashboard page a status of usable, restricted, provisional or withheld, with rules for uncertainty, revisions and rechecks.
A measurement dashboard page that always looks the same gives readers no warning when its numbers weaken. Four statuses fix that: usable, restricted, provisional and withheld. Each says what the reader may do with the page today, and each changes when a definite, checkable thing changes rather than when someone feels less sure.
What to take away
- Every page carries one statususable, restricted, provisional or withheld.
- The status names the permitted action, so a reader knows what the number may be used for.
- Uncertainty that crosses the action threshold restricts or withholds the decision; it is not left in a tooltip.
- Provisional data is marked in headings and downloads, not only in colour.
- Loss of lineage, an inaccessible critical path, an absent denominator or an unexplained reconciliation difference withholds the affected page.
Give every page one of four statuses
Put the status in the header, beside the decision owner, the question, the permitted action and the next review date. The newest limitation sits next to it. A red or green tile alone is ambiguous and inaccessible to some users, so the status is always written as a word.
| Status | What the reader may do | Show beside it |
|---|---|---|
| Usable | Act on the number for the named decision | Newest limitation and next review date |
| Restricted | Use it only for the narrower action the page states | The uncertainty or gap that restricts it |
| Provisional | Treat it as open to later change | Which later inputs are awaited |
| Withheld | Do not act on the affected value | Why it was withheld and who owns the fix |
Dashboard header essentials
- Name decision owner and question
- State permitted action
- Set next review date
- Show statususable, restricted, provisional, withheld
- Display newest limitation beside status
The page owner sets the status, not the charting tool. Write the rule for each status once, in the decision record, so the label does not drift with the mood of a review meeting. A commercial dashboard needs its own decision rules and should not imply public-service assessment.
Restrict the decision when uncertainty reaches the threshold
Restrict or withhold the decision when uncertainty crosses the action threshold; do not hide the interval in a tooltip. Place material coverage gaps, sampling intervals, model sensitivity and reporting lag next to the affected value. Say how each one changes the way the number may be used.
The Government Analysis Function makes the same case. Being upfront about quality, uncertainty and change helps users avoid conclusions that the statistics do not support. It was written for government statistics, so a private team borrows the principle, not an obligation.
Label the kind of value as well. Observed data stays separate from imputed, adjusted, modelled or attributed output, and an attribution column says "credited outcomes", not "outcomes caused". A restricted page that shows a modelled value should say so next to the figure, not in a footnote.
Mark provisional data and keep every revision
Mark provisional data in headings and downloads, not only in colour. A provisional status means later inputs are expected, so the number may change without anyone having made an error. That planned revision is different from the correction of an error, and the log should record which one occurred.
Store original and revised values with the date, reason, affected period, absolute change and decision effect. The log lets a reader see whether a restated figure moved a decision, which a silent overwrite hides. Withholding is a status, not a deletion, so the earlier values and the reason stay on record.
If a correction reverses an action, reopen the decision record and inspect dependent charts. If it reveals a shared transformation fault, withhold related periods until revalidation. The same rule covers every downstream page that reuses the faulty transformation, even when its own figures look sensible.
Withhold the page when the evidence breaks
Withhold the affected page, not the whole dashboard, when the evidence behind a number breaks. Four failures trigger it, and each is a fact someone can check, so the switch needs no debate. A withheld page stays visible, with its reason, so readers know why the number has gone.
- Lineage losta value has no link or identifier back to its collection specification, source extract, transformation, reconciliation and approval.
- Critical path inaccessiblea reader cannot reach the decision or its caveat, for example by keyboard.
- Denominator absentsuppress the output rather than substituting traffic or another convenient total, as the reporting mistakes guide warns.
- Reconciliation unexplainedthe gap between measurement and the operational source has no owner and no explanation, a problem that reconciliation coverage is meant to expose.
Metric card fields
- Event, numerator, denominator, population, unit
- Period, time zone, inclusions, exclusions
- Data owner and caveat
- Source-system and calculation versions
- Last refresh and next expected update
Each displayed measure opens a metric card with event, numerator, denominator, population, unit, period, time zone, inclusions, exclusions, data owner and caveat. That card is where an absent denominator shows first. The measurement register sets out how such cards fit together.
Whatever the status, the page shows where its numbers came from. The Government Analysis Function's dashboard guidance says to include metadata in a dashboard, covering links to the source data, the last update and the date of the next update.
Recheck after every change
Stop or recheck after a changed definition, source, permission route, model, geography or user need. Any one of these can turn a usable page into a restricted one, so review the status at each change and at the header review date. A status nobody revisits is only a label.
Test the statuses with intended readers. Give them a realistic question and ask what they would do, which population they think the number covers and whether it is observed or modelled. A technically correct page fails if its readers infer the wrong action from the status.
Check that the statuses survive assistive use, including keyboard navigation, focus order, heading structure, filter announcements, chart alternatives, zoom, small screens and error states. A readable data table backs up each chart. Reporting lag and freshness limits belong in the analytics service standards.
Common questions
What does each dashboard status mean?
Usable means the number may support the named decision. Restricted means it supports only the narrower action the page states. Provisional means later inputs are expected. Withheld means the affected value must not be acted on until an owner resolves the fault.
What turns a usable page into a withheld one?
Loss of lineage, an inaccessible critical path, an absent denominator or an unexplained reconciliation difference. Any of these switches the affected page to withheld until an owner resolves it, and a changed definition or source forces a recheck first.
Should a revised number overwrite the original?
No. Store the original and revised values with date, reason, affected period, absolute change and decision effect. Distinguish a planned revision caused by later inputs from the correction of an error, and mark provisional data in headings and downloads.
