From a defect to the thing that caused it

A finding is not a cause, and only a cause can be fixed

At this point you have findings. Reporting collapsed to 29% in August. One school under-reported attendance by 42%. One survey team rounds heights to the nearest half centimetre. Twenty-four percent of ages are whole years.

Every one of those is a what. None of them is a why, and a corrective action plan written against a what produces the sentence that appears in most DQA reports in this sector:

Recommendation. Train staff on data quality.

That sentence is written when nobody looked for a cause. It is unfalsifiable, it is expensive, and it will not change August’s reporting rate, because nothing about August was caused by staff not knowing what they were doing.

Five categories, and they need different money

Almost every data quality cause in routine programme data falls into one of five categories. Naming which one you are in tells you who has to fix it.

Category	Looks like	Fixed by
Definition	Two sites counting different things under one indicator name	A written indicator reference sheet
Tool	A form that cannot express what happened, or a system that drops values	A form or configuration change
Capacity	The right tool used wrongly, consistently, by someone who was never shown	Supervision and job aids, occasionally training
Incentive	Reporting that improves when it is looked at, or numbers that meet a target exactly	Changing what the number is used for
System	Everyone fails at once — no forms, no fuel, no network, no supervisor	Logistics, budget, staffing

Two observations about that table, and both come up in every assessment.

Definition is the largest category and the least suspected. A verification factor far from 1 is more often two people counting different things than anyone counting wrongly. It is also the cheapest to fix, and the fix — writing the definition down — prevents recurrence permanently.

Capacity is the smallest category and the most frequently blamed, because it is the only one where the fix is a workshop and a workshop is procurable.

Let the data narrow it before you ask anyone

The pattern of a defect tells you which category it is in, and you already have the data to see the pattern.

by_month = vax.groupby(vax["period"].dt.strftime("%Y-%m"))["reported"].mean()
by_facility = vax.groupby("facility_id")["reported"].mean()

print("spread across months:  ", round(by_month.max() - by_month.min(), 2))
print("spread across facilities:", round(by_facility.max() - by_facility.min(), 2))

by_month <- vax |> mutate(m = format(period, "%Y-%m")) |>
  summarise(r = mean(report_submitted), .by = m)
by_facility <- vax |> summarise(r = mean(report_submitted), .by = facility_id)

c(months = diff(range(by_month$r)), facilities = diff(range(by_facility$r)))

Three shapes, three categories:

• Concentrated in time, spread across units. August, everywhere. That is a system cause — something failed for the district at once. Twenty-seven facility findings would be twenty-seven wrong answers.
• Concentrated in one unit, spread across time. SCH09’s Y/N coding, every month. That is a tool or definition cause local to one site.
• Spread across both. Age heaping, every team, all year. That is a method cause, inherent to how the measurement is taken at all.

Write the shape down before you propose a cause. It rules out three of the five categories for free, and it is the step that stops a system failure being written up as twenty-seven capacity failures.

Then ask the form

For anything local, look at the instrument before you look at the person.

• What values can the form actually accept? SCH09’s registers offered Y/N boxes; the extract’s boolean cast recognised true/false. Neither the teacher nor the analyst did anything wrong, and no training fixes it.
• Is the field required? An optional field with a 40% missing rate is a form design decision, not a compliance problem.
• What does the field label say? A column headed “cases” collects whatever the person filling it believes a case is.

The five whys, with a stopping rule

The technique is standard. What is usually missing is a rule for when to stop, which is why five whys so often terminates at “staff are not motivated”.

Stop when you reach something a named person can change with a budget you can estimate. Anything past that point is philosophy.

Attendance was under-reported by 42% in SCH09. — Why? The extract read Y as missing. — Why? The register used Y/N and the form expected true/false. — Why? The paper register printed in 2019 has yes/no boxes; the digital form was designed later, separately. — Stop. The education office can reprint the register or the HMIS team can accept both codings. Both are costed in an afternoon.

Going further — why were they designed separately? — produces true statements about institutional coordination that no one in this assessment can act on.

Test the cause against the data

A proposed cause makes a prediction. Check it before you write it down; it takes one query and it is the difference between a finding and a story.

# Cause: one school's register uses Y/N. Prediction: Y/N appears in that
# school only, and in most of its months.
yn = attendance[attendance["present"].isin(["Y", "N"])].merge(
    roster[["student_id", "school_id"]], on="student_id", how="left"
)
print(yn["school_id"].value_counts())
print(yn.groupby("school_id")["attendance_date"].nunique())

attendance |>
  filter(present %in% c("Y", "N")) |>
  left_join(select(roster, student_id, school_id), by = "student_id") |>
  count(school_id)

If the Y/N values turned out to be spread across six schools, the cause is not one register — it is a regional convention, and the fix is different. A cause that survives its own test is worth writing; one that was never tested is a guess in the voice of a conclusion.

Put the cause at the level the fix lives at

The last discipline, and it decides whether the report is fair.

• A facility-level cause needs a facility-level action. “FAC020’s tally sheet is missing” is fixed by delivering a tally sheet.
• A district-level cause needs a district-level action. “No facility received forms in August” is not thirty-eight findings.
• A system-level cause needs a system-level action, and naming it is often the most valuable thing in the report even though nobody in the room can fix it today.

Attributing a district-level cause to facilities is the single most damaging error in DQA writing. It produces defensive reporting, the numbers get smoothed, and next quarter’s assessment finds a cleaner dataset that is less true.

The purpose of naming causes is that the next round is better. A report that assigns blame accurately and changes nothing has failed at the only thing it was for.

What comes next

You have findings, and now each has a cause with a level and a test behind it. The last lesson is the document — what goes in it, in what order, and the four columns that turn a finding into something that is still true when somebody checks in three months.