Outlook
Part of What is actually known about measurement going into 2027, and what still is not
Test measurement skills with a practical exercise, not a certificate
Check measurement capability with a one-event practical exercise across four skill bundles, then record the result in a capability matrix, not a job title.
A certificate tells you what someone studied, not what they can do with a broken measurement set-up. A short practical exercise shows it directly. Set one task for each of four capability bundles, record what the person produced, and note what was left untested before anyone is trusted with a release.
What to take away
- Competence shows in work producta test journey, a map, a note, a rollback, not a badge.
- Four bundles cover permission analysis, measurement engineering, statistical assurance and secure AI operation.
- One event traced from permission state to report touches every bundle.
- Untested items are findings, not passes, and the record names them.
- The result belongs in a capability matrix, not in a new job title.
Test the work, not the credential
Competence shows up in what a person produces. A measurement engineer proves it with an agreed test journey and a change record, not a tool badge. A permission analyst proves it with a map, not a training certificate. An England employer should still require a qualified adviser for legal conclusions.
Keep the exercise bounded to one event. A single event traced from permission state to report touches every bundle, so the evaluator sees the whole chain without a day of testing. Do not include real personal data in recruitment or training samples.
Set one task for each of four bundles
The four bundles come from the work, not from job titles. Each row below gives the task to set, the evidence to ask for and the thing that does not count as proof.
| Bundle | Task to set | Evidence to ask for | Does not count |
|---|---|---|---|
| Permission and purpose analysis | Map storage or access, purpose, data category, recipient, retention and objection or consent state | A map that keeps PECR questions apart from UK GDPR processing | A training certificate as legal authority |
| Measurement engineering | Define events, control schemas, run state-aware tests and reconcile source to report | An agreed test journey and a change record | A tool badge |
| Statistical assurance | State population, numerator, denominator, unit, period, time zone, exclusions and uncertainty | A note that separates observed from modelled values and revisions from errors | A chart without its definitions |
| Secure AI operation | Document model and data dependencies, threat assumptions, supplier access, monitoring, incident route and retirement conditions | A written route from alert to retirement | A model name without its dependencies |
Four capability bundles
Permission and purpose
- Core output
- purpose and consent map
- Evidence of competence
- qualified adviser sign-off
- Key standard
- PECR separate from UK GDPR
- Not sufficient
- training certificate
Measurement engineering
- Core output
- agreed test journey
- Evidence of competence
- change record
- Key standard
- rollback of faulty release
- Not sufficient
- tool badge
Statistical assurance
- Core output
- uncertainty note
- Evidence of competence
- population and denominator
- Key standard
- AQuA proportional assurance
- Not sufficient
- observed value alone
Secure AI operation
- Core output
- dependency and threat log
- Evidence of competence
- monitoring and incident route
- Key standard
- NCSC secure AI guidance
- Not sufficient
- in-house only assumption
Permission analysis is the bundle where the questions of advertising rules meet the page, because it asks what fires before a person chooses. Ask the analyst to explain the answer in the terms of the map, and to say which parts need a qualified adviser.
Give the statistical and AI tasks a standard
The statistical task needs a standard to mark against. The Analytical Quality Assurance Book provides guidance on good quality analysis for government. It covers proportional assurance, validation, uncertainty and continuing review, including black-box analysis, so a candidate can be asked what level of checking a given figure deserves.
For the AI task, the NCSC's secure AI system development guidelines are aimed at providers of AI systems that use hosted models or external APIs. The guidance spans design, development, deployment and operation. Pair it with the bounded AI uses when you decide what the candidate may and may not touch.
Run the one-event exercise
Run the same four steps for every candidate, so that results can be compared. The evaluator holds the answers back until the end, and the candidate works from the same event and the same documents.
- Trace one event from permission state to report, naming each system it passes through.
- Introduce a known schema error and ask the candidate to find it and explain the effect on the report.
- Ask for an uncertainty note on the resulting figure, stating what the number can and cannot support.
- Rehearse rollback of the faulty release, following the controlled release workflow, and confirm that the report returns to a known state.
The exercise can sit inside a wider ninety-day test plan, where each candidate's output feeds a later decision rather than a one-off impression.
Record what ran and what did not
Record which tests ran and which remain untested. An untested item is a finding, not a pass, so the record names it plainly. Ask the candidate to say when legal, technical or statistical uncertainty should halt release, because escalation judgement cannot be inferred from a software credential.
Some uncertainties show up only in rehearsal, such as a silent platform change or a decision that later reverses. The measurement risks to rehearse give prompts for that conversation, and the answers go on the same record as the test results.
Put the result in a capability matrix
Begin with a capability matrix, not a new job title. Put each production task down the left and the four bundles across the top. Name the accountable person, the evidence they can produce, a deputy, a review date and any unresolved gap.
Capability matrix staffing steps
- List production tasks down the left
- Place four bundles across the top
- Name accountable person and deputy
- Record evidence and review date
- Flag unresolved gaps
- Keep challenge independent where risk is high
One person may cover several cells, but privacy, statistical and security challenge should keep genuine independence where risk is high. The six delivery roles show how authority can be shared out across the matrix once the exercise has shown who can do what.
Increase specialist capacity when defined triggers appear, such as repeated reconciliation failures, an unreviewed modelling change, an approaching service-level breach or no independent approver. Reduce or redirect it when the relevant work retires. Named privacy, legal, security, statistical and employment reviewers should sign the matrix.
Common questions
Why test a practical exercise instead of checking certificates?
A certificate shows what someone studied. The exercise shows the work product: a test journey, a map, an uncertainty note and a rehearsed rollback. Competence is demonstrated with an agreed test journey and change record, not a tool badge.
What should the evaluator record after the exercise?
Which tests ran and which remain untested, plus the candidate's answer on when legal, technical or statistical uncertainty should halt release. Untested items are findings, not passes, and they go into the capability matrix as unresolved gaps.
When should specialist capacity be increased or reduced?
Increase it when defined triggers appear, such as repeated reconciliation failures, an unreviewed modelling change, an approaching service-level breach or no independent approver. Reduce or redirect it when the relevant work retires.
