Back to the lesson·Lesson 6 of 8·Finding the sites worth visiting
What the last digit tells you
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- Measurements have a texture
- The check
- Age heaping, and why it is usually nobody's fault
- Calibrate against what chance produces
- Why the null case matters more
- What you may write
- What comes next
Speaker notes
Digit preference, heaping and the calibration that separates a finding from noise. One team's heights end in .0 or .5 sixty-nine percent of the time; the district's most suspicious facility turns out to be chance.Measurements have a texture
- A number produced by reading an instrument has a particular texture: the last digit is close to uniform, because the true value is equally likely to fall anywhere within the smallest division of the scale.
Speaker notes
A number produced by reading an instrument has a particular texture: the last digit is close to uniform, because the true value is equally likely to fall anywhere within the smallest division of the scale. A number produced by estimating, rounding, or recalling has a different texture. It clusters on fives and zeros, on whole years, on multiples of ten. That clustering is digit preference, and it is measurable without going anywhere. It is the integrity dimension's main tool, and this lesson spends as much time on what it cannot show as on what it can.The check — In Python
import pandas as pd smart = pd.read_csv("smart-nutrition-survey-2024.v1.csv") smart["terminal"] = (smart["height_cm"] * 10).round() % 10 by_team = ( smart.assign(rounded=smart["terminal"].isin([0, 5])) .groupby("team") .agg(rounded_share=("rounded", "mean"), n=("rounded", "size")) ) print((by_team["rounded_share"] * 100).round(0))The check — In R
library(dplyr) smart |> mutate(terminal = round(height_cm * 10) %% 10, rounded = terminal %in% c(0, 5)) |> summarise(rounded_share = mean(rounded), n = n(), .by = team)The check
- That is not a marginal result and it does not need a test — Nothing about a population makes one team's children's…
Speaker notes
Two of ten possible terminal digits is 20% under no preference. Teams 1, 3 and 4 sit at 18%, 18% and 23% — which is what measurement looks like. Team 2 sits at 69%. That is not a marginal result and it does not need a test. Nothing about a population makes one team's children's heights land on half-centimetres three and a half times as often as everybody else's. It is how the team read the board: rounding to the nearest half centimetre instead of reading the millimetre.Age heaping, and why it is usually nobody's fault — In Python
ages = smart["age_months"].dropna() whole_years = (ages % 12 == 0).mean() print(f"{whole_years:.0%} of ages fall on an exact multiple of 12 months") print(ages.value_counts().reindex([22, 23, 24, 25, 26]))Speaker notes
The same phenomenon on ages, where it is nearly universal in this sector.Age heaping, and why it is usually nobody's fault — In R
smart |> filter(!is.na(age_months)) |> summarise(whole_years = mean(age_months %% 12 == 0)) smart |> count(age_months) |> filter(age_months %in% 22:26)Age heaping, and why it is usually nobody's fault
Age in months 22 23 24 25 26 Children 21 15 66 19 17 Speaker notes
Sixty-six children at exactly twenty-four months against fifteen and nineteen either side, and eighty at thirty-six against twenty-eight and seventeen. Across the survey, 24% of ages fall on an exact whole year, against about 9% expected if ages were spread evenly. This matters for a specific, concrete reason and not as a curiosity. The WHO growth standards switch reference table at 24 months, and the SMART age groups are bounded at 12-month marks — so heaping puts a large clump of children exactly on the boundaries where classification changes. And the cause is almost never carelessness. Birth certificates are not universal; a caregiver asked how old a child is will answer "two years"; a local events calendar helps but has limits. Write it up as a finding about the age ascertainment method, with a corrective action about calendars and probing, not as an enumerator error.Calibrate against what chance produces — In Python
reported = vax[vax["reported"]].copy() base_rate = (reported["doses_administered"] % 10 == 0).mean() by_facility = ( reported.assign(round_number=reported["doses_administered"] % 10 == 0) .groupby("facility_id") .agg(rounds=("round_number", "sum"), n=("round_number", "size")) ) by_facility["share"] = by_facility["rounds"] / by_facility["n"] print(f"district base rate: {base_rate:.3f}") print(by_facility.nlargest(3, "share"))Speaker notes
Now the part that separates a usable check from an accusation generator. Run the round-number check on the vaccination extract:Calibrate against what chance produces — In R
base_rate <- mean(vax$doses_administered[vax$report_submitted] %% 10 == 0) by_facility <- vax |> filter(report_submitted) |> summarise(rounds = sum(doses_administered %% 10 == 0), n = n(), .by = facility_id) |> mutate(share = rounds / n) |> arrange(desc(share))Calibrate against what chance produces — In Python
from scipy.stats import binomtest worst = by_facility.nlargest(1, "share").iloc[0] p = binomtest(int(worst["rounds"]), int(worst["n"]), base_rate, alternative="greater").pvalue print(f"p = {p:.4f}, Bonferroni across 38 facilities: {min(1, p * 38):.3f}")Speaker notes
The district base rate is 10.4% — exactly what you would expect. The worst facility, FAC020, reports a round number 14 times out of 60, or 23%. More than double the district rate. On its own that looks like a finding. Test it:Calibrate against what chance produces — In R
worst <- by_facility[1, ] p <- binom.test(worst$rounds, worst$n, base_rate, alternative = "greater")$p.value c(p = p, bonferroni = min(1, p * 38))Calibrate against what chance produces
- p = 0.003 on its own, and 0.11 after correcting for having looked at thirty-eight facilities — Three facilities fall…
Speaker notes
p = 0.003 on its own, and 0.11 after correcting for having looked at thirty-eight facilities. Three facilities fall below p = 0.05 uncorrected, and 1.9 are expected by chance. Three against 1.9 is nothing. There is a second reason to let it go. FAC020's monthly doses run from five to thirteen, so "ends in zero" almost always means the value was exactly ten. On counts that small, the check has no power to distinguish rounding from arithmetic.Calibrate against what chance produces
The result is: no evidence of digit preference in the vaccination reporting. That is a real finding, it took four lines, and it is the one you should hope for.
Why the null case matters more
- Compute the base rate from the data, not from theory. Terminal digits are not uniform when counts are small, when a…
- Correct for the number of units you looked at. Testing thirty-eight facilities at 5% produces about two positives…
- Require a mechanism. Team 2's 69% has one — the team read the measuring board to the nearest half centimetre. If…
Speaker notes
Most of what makes this check dangerous is that it is almost always run on data where nothing is wrong, by someone hoping to find something. Three rules that keep it honest.What you may write
Do not write Write "FAC020's data appear fabricated" "FAC020 reports round numbers more often than the district average; the difference is not significant after adjusting for the number of facilities examined" "Team 2's measurements are unreliable" "Team 2's height readings end in .0 or .5 in 69% of cases against 18–23% for other teams, consistent with reading the board to the nearest half centimetre" "Ages are inaccurate" "24% of ages fall on an exact whole year against 9% expected, indicating age was estimated rather than documented for a substantial share of children" Speaker notes
The wording is not a courtesy; it decides what happens next. The right-hand column says what was observed, how large it is, and what mechanism it is consistent with. The left-hand column says what somebody did. Only one of the two produces a corrective action, and only one of them survives being shown to the team it describes.What comes next
- You now have findings — a completeness collapse, a verification factor of 1.42, a team rounding its heights.
Speaker notes
You now have findings — a completeness collapse, a verification factor of 1.42, a team rounding its heights. None of them is yet a cause, and none of them can be acted on. The next lesson is how to get from a defect to the thing that produced it, and why "the staff need training" is almost always the wrong answer.