Fix the definition before building the dashboard

A dashboard cannot settle a disagreement about the facts. It can only make the disagreement look more finished.

If two teams arrive at a meeting with different numbers, the first question concerns what each number represents, where it came from and what happened to the records that did not fit. The chart can wait.

Most reporting failures begin in definition, ownership and operating discipline rather than visualisation.

The order of work is simple:

Define the decision. Define the number. Define where it comes from. Define what is missing. Define who owns it. Then build the dashboard.

A dashboard is the last step, not the first

A dashboard is not a management system. It is an interface to one.

Modern reporting tools make presentation easy. A team can connect several systems, write a few measures and produce something convincing in days. None of this proves that the result means what management thinks it means.

A sales dashboard may correctly count every row marked Closed Won while cancelled orders remain in the population. An operations dashboard may correctly average completion times while silently excluding jobs with no completion timestamp. A maintenance report may show that the dashboard refreshed five minutes ago even though the field system stopped synchronising twelve hours earlier.

The chart has not failed in any of these cases. The definition has.

Peter Drucker made the wider management argument in The Information Executives Truly Need: information should be organised around the requirements of the business, not merely around what data-processing tools can produce. The operational implication is clear. Begin with the decision. Only then decide what deserves to appear on the screen.

The UK government’s AQuA Book draws a useful distinction between verification and validation. Verification asks whether the analysis was built as specified. Validation asks whether it is the right analysis for the intended use. A report can therefore be technically correct and still be managerially wrong.

Trustworthy reporting needs six controls: named sources, visible exceptions, explicit calculations, visible freshness, named ownership and controlled change.

Begin with the management question

Write down the decision the report is meant to support. “Show weekly performance” is too loose. A useful question names the thing being decided.

For example:

• Which planned jobs are ready to proceed?
• Which departments have submitted a complete report for this period?
• Where will planned demand exceed controlled stock?

The wording matters because each question needs a different definition. A job-readiness report must define “ready”. A submission report must distinguish missing, late, duplicated and corrected returns. A stock report must decide how unknown quantities differ from confirmed zero.

State what management will do differently depending on the answer. If the result cannot change a decision, it may not deserve to be a management measure.

When the question remains vague, these choices do not disappear. They are made inside a spreadsheet formula, join or dashboard filter where the people accountable for the work cannot see them.

Name the source for each fact

“From the database” is not a source definition.

Operational reports often combine records from several systems. One may hold the job, another the permit, another the field evidence and another the commercial position. The report should not pretend that one system owns everything.

The familiar idea of one universal source of truth is often too crude. Different systems may legitimately be authoritative for different facts. The CRM may own customer status. The finance system may own invoiced value. The job-management system may own operational completion.

For every important measure, record:

1. the source system and dataset;
2. what one source record represents;
3. the fields or calculation used;
4. how the data arrives and how often it should change;
5. what happens when it is missing or conflicts with another source;
6. who can correct it.

A compact source definition might read:

Source: Job Management System → Work Orders
Authoritative for: operational job status
Business grain: one work order
Extract: API every 30 minutes
Source owner: Head of Operations
Known dependency: field devices must synchronise before status appears centrally

This is a source-authority map. It prevents a convenient copy of a value from quietly replacing the real record.

Where systems disagree, decide which source has authority for that fact or deliberately preserve the disagreement as an exception. Join order, COALESCE() logic or the easiest available table must not make management policy by accident.

The NIST Research Data Framework treats provenance, metadata and data quality as connected concerns. The practical lesson for operational reporting is that source identity and processing history must survive the journey into the dashboard.

The SIREN portfolio record shows this principle in a working system: sources remain separate, their health is visible and an incomplete evidence chain cannot produce a confident answer. It is private Strataflow portfolio work, not a published client outcome.

Keep exceptions in the report

Operational reporting becomes dangerous when cleaning the data means cleaning away the operation.

Missing identifiers, duplicate jobs, conflicting statuses, failed joins and records arriving after the reporting cut-off are not merely technical irritations. They are evidence about how the process is working.

The Government Data Quality Framework separates completeness, uniqueness, consistency, timeliness, validity and accuracy. These are different problems and should not be collapsed into a vague statement that the data has been “cleaned”.

Decide explicitly how each class is handled:

• Missing: Is the value genuinely absent, not yet received or not applicable?
• Duplicate: Are both records retained, and which one is used in the summary?
• Conflicting: Which source has authority, and does the disagreement need action?
• Corrected: Is the original preserved, and can the change be explained?
• Stale: How old can the evidence be before the report stops making a claim?

Keep exclusions and exceptions separate. An exclusion is intentionally outside the definition: cancelled jobs may reasonably be excluded from an on-time completion measure. An exception is a record that should have been classifiable but was not. Exclusions belong in the metric definition. Exceptions belong beside the result.

The result alone is insufficient:

Completed on time: 94.2%

Report enough context to interpret it:

Completed on time: 94.2%
Eligible jobs: 8,412
Reporting exceptions: 137
Missing completion timestamp: 82
Conflicting status: 31
Unmatched job reference: 24

The percentage has not changed. Its meaning has.

Exception counts also provide early warning. Rising unmatched jobs may show that an integration is breaking. More duplicates may expose a failing identifier policy. Missing completion timestamps may point to a field-app fault, a training problem or a gap in the workflow.

In many operations, the exception report is the management report.

Define the calculation in ordinary language

A metric name is not a metric definition.

“Jobs completed”, “utilisation”, “conversion”, “backlog”, “first-time fix” and “active customer” all sound obvious until two competent people calculate them differently.

Write the rule before writing the query. A useful definition might say:

A job is ready when the linked permit is granted, the planned work date falls inside the permit dates and the permit record has been refreshed within the agreed period.

That sentence can be reviewed by the people who own the work. It can then become a tested query. If the business rule changes, the report, test and operating note should change together.

For an important measure, use a metric contract:

The SQL, DAX, Python or BI expression should implement this definition. It should not be the only place the definition exists.

For a critical figure, take ten to twenty representative records and follow each one from the source to the reported answer. Let a manager and analyst inspect them manually. If a small sample cannot be reconciled, adding millions of rows will not make the metric more trustworthy.

Make freshness visible

“Live dashboard” is not an adequate description of freshness.

Freshness is a business requirement. Five-minute-old data may be unnecessary for monthly finance reporting and dangerously stale for dispatch operations. The Government Data Quality Framework therefore defines timeliness in relation to the data’s intended use.

Where the architecture has separate stages, show separate timestamps:

Source data current to: 25 Aug 2026, 14:32 BST
Dashboard refreshed: 25 Aug 2026, 14:41 BST

They do not prove the same thing.

Microsoft’s Power BI refresh guidance distinguishes source-data refresh, file synchronisation and query-cache or dashboard-tile refresh. A report can therefore have recent presentation activity without proving that its original source is current.

Set an operational threshold and display the result against it:

Expected: less than 60 minutes old
Actual: 9 minutes
Status: Current

If a source refresh fails, preserve the last known result only when its age and status are obvious. Do not continue showing yesterday’s number with today’s browser date.

Give the number an owner

Every important number needs a person with the authority to answer for its meaning.

This is not the same as asking who built the dashboard.

A practical division is:

• Metric owner: owns what the measure means and whether it remains useful.
• Data owner: owns the quality and interpretation of the source data.
• Technical owner: owns the pipeline, reporting model and dashboard operation.

In a small organisation one person may hold more than one role. The point is to make the responsibilities explicit.

The UK government’s data ownership model similarly separates accountability for the quality and integrity of data from ownership of a platform or service.

Without named owners, exceptions become an inbox shared by everybody and owned by nobody. The dashboard can display a blockage, but it cannot decide who must resolve it.

When somebody asks why a KPI changed, the answer should not be “ask whoever built the report”.

Control changes to the definition

Definitions change because businesses change. That is normal. Undocumented change is not.

Perhaps “active customer” originally meant anyone invoiced within twelve months and later changes to six. Perhaps a new job type becomes eligible for the service level. Perhaps operational status moves from one application to another.

Record the old definition, new definition, effective date, owner, reason and whether historical data has been restated. Mark any break in comparability on the trend itself.

Otherwise the organisation can spend months explaining a performance movement that was actually a definition movement.

What goes wrong in practice

Most dashboard failures happen before the visual layer.

A report is not declared trustworthy once and forever. It remains trustworthy while its definition, sources, assumptions, exceptions and ownership remain under control.

A practical operating method

Begin with a management question rather than “What charts would you like?”

What decision are we trying to make?

Then work in this order:

1. Define the decision. State what management will do differently depending on the answer.
2. Name the owner. Identify who controls the metric’s meaning and who owns the source data.
3. Define the population. State what one record represents and what the complete eligible set should contain.
4. Settle source authority. Decide which system owns each required fact.
5. Define exclusions and exceptions. Say what is intentionally outside the measure and what should have been classifiable but was not.
6. Write the arithmetic. Record the numerator, denominator, dates, filters, joins, deduplication, rounding and null handling in ordinary language.
7. Set the freshness requirement. Define how old each source may be for the decision being made.
8. Reconcile real cases. Follow a small sample from source record to reported result and resolve every unexplained difference.
9. Build and test the reporting model. Test calculations, joins, refresh failure and exception handling against the agreed definitions.
10. Publish the controls with the result. Show the source, freshness, exceptions, definition version and owner where management can find them.
11. Monitor change. Watch exception volumes, refresh failures and changes to the source or metric definition.

This order deliberately postpones visual design. That is what makes the visualisation useful when it arrives.

A good operational report should expose enough of its workings to be challenged:

Metric: On-time completion
Result: 94.2%
Source: Job Management System
Source current to: 14:32 BST
Reporting exceptions: 137
Definition: v1.3
Owner: Operations Director

This small block does more for trust than another gauge or colour-coded tile.

It also changes the meeting. Instead of asking where the number came from, management can ask why missing completion dates rose and who is correcting the cause.

That is the purpose of operational reporting: shorten the argument about evidence so attention can move to the decision.

A meeting-readiness check

An organisation does not need a large data-governance programme before improving its reporting.

Choose one dashboard that matters and apply five tests:

1. Write one important KPI definition without referring to the dashboard code.
2. Put the source and owner beside the definition.
3. Count the records the dashboard cannot classify.
4. Show when the underlying source was actually current.
5. Reconcile ten to twenty real records from source to dashboard.

The immediate objective is one important number that the responsible manager can explain.

Once that standard works, repeat it. The organisation gradually acquires something more useful than a collection of dashboards: a controlled reporting system in which numbers have definitions, definitions have owners, sources retain their authority, exceptions remain visible and changes leave a record.

Operational Data & Reporting is for reports that arrive late, disagree with another system or need manual repair before a meeting. The work starts with the management question and the source records, not with a new dashboard.