Five dimensions, five measures

The sentence to stop saying

“The data quality is poor.” Everyone nods, nothing happens, and the same sentence is said next quarter.

It fails because it is unactionable in three separate ways. It does not say which quality — a late report and a fabricated report are different problems with different fixes. It does not say how much — poor everywhere or poor in four facilities. And it does not say so what — whether the defect is large enough to change the number anyone is going to act on.

The five dimensions exist to fix all three at once. They are not a framework to recite; they are five questions, each of which has a number as its answer.

The five, and the measure for each

Dimension	The question	The measure
Completeness	Did everyone report, on everything?	Reporting rate; missing-value rate per field
Timeliness	Did the report arrive in time to be used?	Share submitted by the deadline; median days late
Accuracy	Does the reported figure match the source?	Verification factor: recount over reported
Consistency	Do the numbers agree with each other and with last month?	Rate of failed internal rules; outlier rate
Integrity	Was the value produced by measurement or by something else?	Digit preference; heaping; implausibly stable series

Two things about that table are worth arguing over, because both come up.

Accuracy is the only one that needs the source document. The other four are computable from the extract you already have, at your desk, this afternoon. That asymmetry drives the whole design of a DQA: you use the four desk dimensions to decide where to spend the expensive fifth.

Integrity is not an accusation. It measures whether numbers look like measurements. A heaped age distribution usually means nobody asked for a birth certificate, not that anybody invented anything, and the lesson on it spends most of its time on that distinction.

Compute all five on one extract

Take the routine vaccination extract — 38 facilities, twelve months, six antigens.

import pandas as pd

vax = pd.read_csv("vaccination-coverage-2024.v1.csv", parse_dates=["period"])
vax["reported"] = vax["report_submitted"] == True

scorecard = {}

# Completeness
scorecard["reporting_rate"] = vax["reported"].mean()
scorecard["field_completeness"] = 1 - vax.isna().mean().mean()

# Consistency: doses cannot exceed the target population
rule_fail = (vax["doses_administered"] > vax["target_population"]).mean()
scorecard["rule_failure_rate"] = rule_fail

# Integrity: share of reported values ending in zero
reported = vax[vax["reported"]]
scorecard["round_number_share"] = (reported["doses_administered"] % 10 == 0).mean()

print(pd.Series(scorecard).round(3))

library(dplyr)
library(readr)

vax <- read_csv("vaccination-coverage-2024.v1.csv")

scorecard <- tibble::tibble(
  reporting_rate     = mean(vax$report_submitted),
  field_completeness = 1 - mean(is.na(as.matrix(vax))),
  rule_failure_rate  = mean(vax$doses_administered > vax$target_population),
  round_number_share = mean(
    vax$doses_administered[vax$report_submitted] %% 10 == 0
  )
)

round(scorecard, 3)

Measure	Value	Read as
Reporting rate	76.5%	642 of 2,736 facility-months carry no report
Field completeness	100%	no blank cells — which is not the same as no missing data
Rule failure rate	0%	no facility reports more doses than its target population
Round-number share	10.4%	what chance produces; nothing to see

Look at rows two and one together. Field completeness is 100% and reporting completeness is 76.5%, and only the second one matters. Every non-reporting facility-month is present in the file as a row of zeroes with a flag, so a blank-cell count says the data is perfect. This is the defect the whole module keeps circling: a missing report that arrives as a zero.

Weight the dimensions by what they cost

A scorecard with five equal numbers implies the five matter equally. They do not, and which matters most depends entirely on what the data is for.

• For a coverage figure, completeness dominates. A 76.5% reporting rate makes any district total a lower bound, and no amount of accuracy in the reports you did receive fixes that.
• For a caseload figure used for procurement, accuracy dominates. Ordering therapeutic food against an over-reported caseload wastes money; against an under-reported one, children go without.
• For an early warning indicator, timeliness dominates. A perfectly accurate report arriving six weeks after the outbreak has zero value.

Say which dimension you weighted and why, in the report. A composite quality score that hides the weighting is the same defect as an indicator that hides its denominator.

Do not collapse it to one number

Somebody will ask for a single score out of 100. Resist, and offer the scorecard instead, for a specific reason: the five dimensions have different fixes, and the single number tells you nothing about which one to apply.

A district at 82% because it is late is fixed by moving a deadline. A district at 82% because a third of its facilities never report is fixed by finding out why they stopped. Both score 82. Only one of them is fixed by a training workshop, and neither is fixed by the workshop somebody will propose.

If a composite is genuinely required — some donor templates demand it — publish it beside the components, never instead of them.

scorecard_table = pd.DataFrame({
    "dimension": ["Completeness", "Timeliness", "Accuracy", "Consistency", "Integrity"],
    "measure": ["Reporting rate", "Submitted by deadline", "Verification factor",
                "Rule failure rate", "Round-number share"],
    "value": [0.765, None, None, 0.0, 0.104],
    "source": ["extract", "submission log", "facility visit", "extract", "extract"],
})
print(scorecard_table)

scorecard_table <- tibble::tribble(
  ~dimension,     ~measure,                ~value, ~source,
  "Completeness", "Reporting rate",         0.765, "extract",
  "Timeliness",   "Submitted by deadline",     NA, "submission log",
  "Accuracy",     "Verification factor",       NA, "facility visit",
  "Consistency",  "Rule failure rate",       0.000, "extract",
  "Integrity",    "Round-number share",      0.104, "extract"
)

The source column is the useful one. It says immediately which numbers you can produce today and which need someone to travel, and that is the shape of every DQA plan.

What comes next

Two of the five dimensions are almost never reported at all, and both are computable from data you already hold. The next lesson computes them properly — reporting completeness and timeliness, each with its own denominator, and the trap of dividing by the facilities that reported.