Lesson 6 of 8
Unit · Finding the sites worth visiting
What the last digit tells you
Digit preference, heaping and the calibration that separates a finding from noise. One team's heights end in .0 or .5 sixty-nine percent of the time; the district's most suspicious facility turns out to be chance.
Measurements have a texture
A number produced by reading an instrument has a particular texture: the last digit is close to uniform, because the true value is equally likely to fall anywhere within the smallest division of the scale.
A number produced by estimating, rounding, or recalling has a different texture. It clusters on fives and zeros, on whole years, on multiples of ten. That clustering is digit preference, and it is measurable without going anywhere.
It is the integrity dimension’s main tool, and this lesson spends as much time on what it cannot show as on what it can.
The check
import pandas as pd
smart = pd.read_csv("smart-nutrition-survey-2024.v1.csv")
smart["terminal"] = (smart["height_cm"] * 10).round() % 10
by_team = (
smart.assign(rounded=smart["terminal"].isin([0, 5]))
.groupby("team")
.agg(rounded_share=("rounded", "mean"), n=("rounded", "size"))
)
print((by_team["rounded_share"] * 100).round(0))
library(dplyr)
smart |>
mutate(terminal = round(height_cm * 10) %% 10,
rounded = terminal %in% c(0, 5)) |>
summarise(rounded_share = mean(rounded), n = n(), .by = team)
| Team | Heights ending .0 or .5 | Children measured |
|---|---|---|
| 1 | 18% | 217 |
| 2 | 69% | 248 |
| 3 | 18% | 248 |
| 4 | 23% | 217 |
Two of ten possible terminal digits is 20% under no preference. Teams 1, 3 and 4 sit at 18%, 18% and 23% — which is what measurement looks like. Team 2 sits at 69%.
That is not a marginal result and it does not need a test. Nothing about a population makes one team’s children’s heights land on half-centimetres three and a half times as often as everybody else’s. It is how the team read the board: rounding to the nearest half centimetre instead of reading the millimetre.
Age heaping, and why it is usually nobody’s fault
The same phenomenon on ages, where it is nearly universal in this sector.
ages = smart["age_months"].dropna()
whole_years = (ages % 12 == 0).mean()
print(f"{whole_years:.0%} of ages fall on an exact multiple of 12 months")
print(ages.value_counts().reindex([22, 23, 24, 25, 26]))
smart |>
filter(!is.na(age_months)) |>
summarise(whole_years = mean(age_months %% 12 == 0))
smart |> count(age_months) |> filter(age_months %in% 22:26)
| Age in months | 22 | 23 | 24 | 25 | 26 |
|---|---|---|---|---|---|
| Children | 21 | 15 | 66 | 19 | 17 |
Sixty-six children at exactly twenty-four months against fifteen and nineteen either side, and eighty at thirty-six against twenty-eight and seventeen. Across the survey, 24% of ages fall on an exact whole year, against about 9% expected if ages were spread evenly.
This matters for a specific, concrete reason and not as a curiosity. The WHO growth standards switch reference table at 24 months, and the SMART age groups are bounded at 12-month marks — so heaping puts a large clump of children exactly on the boundaries where classification changes.
And the cause is almost never carelessness. Birth certificates are not universal; a caregiver asked how old a child is will answer “two years”; a local events calendar helps but has limits. Write it up as a finding about the age ascertainment method, with a corrective action about calendars and probing, not as an enumerator error.
Calibrate against what chance produces
Now the part that separates a usable check from an accusation generator. Run the round-number check on the vaccination extract:
reported = vax[vax["reported"]].copy()
base_rate = (reported["doses_administered"] % 10 == 0).mean()
by_facility = (
reported.assign(round_number=reported["doses_administered"] % 10 == 0)
.groupby("facility_id")
.agg(rounds=("round_number", "sum"), n=("round_number", "size"))
)
by_facility["share"] = by_facility["rounds"] / by_facility["n"]
print(f"district base rate: {base_rate:.3f}")
print(by_facility.nlargest(3, "share"))
base_rate <- mean(vax$doses_administered[vax$report_submitted] %% 10 == 0)
by_facility <- vax |>
filter(report_submitted) |>
summarise(rounds = sum(doses_administered %% 10 == 0), n = n(), .by = facility_id) |>
mutate(share = rounds / n) |>
arrange(desc(share))
The district base rate is 10.4% — exactly what you would expect. The worst facility, FAC020, reports a round number 14 times out of 60, or 23%. More than double the district rate. On its own that looks like a finding.
Test it:
from scipy.stats import binomtest
worst = by_facility.nlargest(1, "share").iloc[0]
p = binomtest(int(worst["rounds"]), int(worst["n"]), base_rate,
alternative="greater").pvalue
print(f"p = {p:.4f}, Bonferroni across 38 facilities: {min(1, p * 38):.3f}")
worst <- by_facility[1, ]
p <- binom.test(worst$rounds, worst$n, base_rate, alternative = "greater")$p.value
c(p = p, bonferroni = min(1, p * 38))
p = 0.003 on its own, and 0.11 after correcting for having looked at thirty-eight facilities. Three facilities fall below p = 0.05 uncorrected, and 1.9 are expected by chance. Three against 1.9 is nothing.
There is a second reason to let it go. FAC020’s monthly doses run from five to thirteen, so “ends in zero” almost always means the value was exactly ten. On counts that small, the check has no power to distinguish rounding from arithmetic.
The result is: no evidence of digit preference in the vaccination reporting. That is a real finding, it took four lines, and it is the one you should hope for.
Why the null case matters more
Most of what makes this check dangerous is that it is almost always run on data where nothing is wrong, by someone hoping to find something.
Three rules that keep it honest.
- Compute the base rate from the data, not from theory. Terminal digits are not uniform when counts are small, when a form has a default value, or when doses come in vials of ten.
- Correct for the number of units you looked at. Testing thirty-eight facilities at 5% produces about two positives from clean data, every time.
- Require a mechanism. Team 2’s 69% has one — the team read the measuring board to the nearest half centimetre. If you cannot name a plausible mechanism, you have a number and not a finding.
What you may write
The wording is not a courtesy; it decides what happens next.
| Do not write | Write |
|---|---|
| “FAC020’s data appear fabricated” | “FAC020 reports round numbers more often than the district average; the difference is not significant after adjusting for the number of facilities examined” |
| “Team 2’s measurements are unreliable” | “Team 2’s height readings end in .0 or .5 in 69% of cases against 18–23% for other teams, consistent with reading the board to the nearest half centimetre” |
| “Ages are inaccurate” | “24% of ages fall on an exact whole year against 9% expected, indicating age was estimated rather than documented for a substantial share of children” |
The right-hand column says what was observed, how large it is, and what mechanism it is consistent with. The left-hand column says what somebody did. Only one of the two produces a corrective action, and only one of them survives being shown to the team it describes.
What comes next
You now have findings — a completeness collapse, a verification factor of 1.42, a team rounding its heights. None of them is yet a cause, and none of them can be acted on. The next lesson is how to get from a defect to the thing that produced it, and why “the staff need training” is almost always the wrong answer.