cassionData Analysis

Back to the lessonLesson 6 of 8Finding the sites worth visiting

What the last digit tells you

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 19

    What this lesson covers

    • Measurements have a texture
    • The check
    • Age heaping, and why it is usually nobody's fault
    • Calibrate against what chance produces
    • Why the null case matters more
    • What you may write
    • What comes next
    Speaker notes
    Digit preference, heaping and the calibration that separates a finding from noise. One team's heights end in .0 or .5 sixty-nine percent of the time; the district's most suspicious facility turns out to be chance.
  2. Slide 2 / 19

    Measurements have a texture

    • A number produced by reading an instrument has a particular texture: the last digit is close to uniform, because the true value is equally likely to fall anywhere within the smallest division of the scale.
    Speaker notes
    A number produced by reading an instrument has a particular texture: the last digit is close to uniform, because the true value is equally likely to fall anywhere within the smallest division of the scale. A number produced by estimating, rounding, or recalling has a different texture. It clusters on fives and zeros, on whole years, on multiples of ten. That clustering is digit preference, and it is measurable without going anywhere. It is the integrity dimension's main tool, and this lesson spends as much time on what it cannot show as on what it can.
  3. Slide 3 / 19

    The check — In Python

    import pandas as pd
    
    smart = pd.read_csv("smart-nutrition-survey-2024.v1.csv")
    
    smart["terminal"] = (smart["height_cm"] * 10).round() % 10
    by_team = (
        smart.assign(rounded=smart["terminal"].isin([0, 5]))
        .groupby("team")
        .agg(rounded_share=("rounded", "mean"), n=("rounded", "size"))
    )
    print((by_team["rounded_share"] * 100).round(0))
  4. Slide 4 / 19

    The check — In R

    library(dplyr)
    
    smart |>
      mutate(terminal = round(height_cm * 10) %% 10,
             rounded = terminal %in% c(0, 5)) |>
      summarise(rounded_share = mean(rounded), n = n(), .by = team)
  5. Slide 5 / 19

    The check

    TeamHeights ending .0 or .5Children measured
    118%217
    269%248
    318%248
    423%217
  6. Slide 6 / 19

    The check

    • That is not a marginal result and it does not need a test — Nothing about a population makes one team's children's…
    Speaker notes
    Two of ten possible terminal digits is 20% under no preference. Teams 1, 3 and 4 sit at 18%, 18% and 23% — which is what measurement looks like. Team 2 sits at 69%. That is not a marginal result and it does not need a test. Nothing about a population makes one team's children's heights land on half-centimetres three and a half times as often as everybody else's. It is how the team read the board: rounding to the nearest half centimetre instead of reading the millimetre.
  7. Slide 7 / 19

    Age heaping, and why it is usually nobody's fault — In Python

    ages = smart["age_months"].dropna()
    whole_years = (ages % 12 == 0).mean()
    
    print(f"{whole_years:.0%} of ages fall on an exact multiple of 12 months")
    print(ages.value_counts().reindex([22, 23, 24, 25, 26]))
    Speaker notes
    The same phenomenon on ages, where it is nearly universal in this sector.
  8. Slide 8 / 19

    Age heaping, and why it is usually nobody's fault — In R

    smart |>
      filter(!is.na(age_months)) |>
      summarise(whole_years = mean(age_months %% 12 == 0))
    
    smart |> count(age_months) |> filter(age_months %in% 22:26)
  9. Slide 9 / 19

    Age heaping, and why it is usually nobody's fault

    Age in months2223242526
    Children2115661917
    Speaker notes
    Sixty-six children at exactly twenty-four months against fifteen and nineteen either side, and eighty at thirty-six against twenty-eight and seventeen. Across the survey, 24% of ages fall on an exact whole year, against about 9% expected if ages were spread evenly. This matters for a specific, concrete reason and not as a curiosity. The WHO growth standards switch reference table at 24 months, and the SMART age groups are bounded at 12-month marks — so heaping puts a large clump of children exactly on the boundaries where classification changes. And the cause is almost never carelessness. Birth certificates are not universal; a caregiver asked how old a child is will answer "two years"; a local events calendar helps but has limits. Write it up as a finding about the age ascertainment method, with a corrective action about calendars and probing, not as an enumerator error.
  10. Slide 10 / 19

    Calibrate against what chance produces — In Python

    reported = vax[vax["reported"]].copy()
    base_rate = (reported["doses_administered"] % 10 == 0).mean()
    
    by_facility = (
        reported.assign(round_number=reported["doses_administered"] % 10 == 0)
        .groupby("facility_id")
        .agg(rounds=("round_number", "sum"), n=("round_number", "size"))
    )
    by_facility["share"] = by_facility["rounds"] / by_facility["n"]
    print(f"district base rate: {base_rate:.3f}")
    print(by_facility.nlargest(3, "share"))
    Speaker notes
    Now the part that separates a usable check from an accusation generator. Run the round-number check on the vaccination extract:
  11. Slide 11 / 19

    Calibrate against what chance produces — In R

    base_rate <- mean(vax$doses_administered[vax$report_submitted] %% 10 == 0)
    
    by_facility <- vax |>
      filter(report_submitted) |>
      summarise(rounds = sum(doses_administered %% 10 == 0), n = n(), .by = facility_id) |>
      mutate(share = rounds / n) |>
      arrange(desc(share))
  12. Slide 12 / 19

    Calibrate against what chance produces — In Python

    from scipy.stats import binomtest
    
    worst = by_facility.nlargest(1, "share").iloc[0]
    p = binomtest(int(worst["rounds"]), int(worst["n"]), base_rate,
                  alternative="greater").pvalue
    print(f"p = {p:.4f}, Bonferroni across 38 facilities: {min(1, p * 38):.3f}")
    Speaker notes
    The district base rate is 10.4% — exactly what you would expect. The worst facility, FAC020, reports a round number 14 times out of 60, or 23%. More than double the district rate. On its own that looks like a finding. Test it:
  13. Slide 13 / 19

    Calibrate against what chance produces — In R

    worst <- by_facility[1, ]
    p <- binom.test(worst$rounds, worst$n, base_rate, alternative = "greater")$p.value
    c(p = p, bonferroni = min(1, p * 38))
  14. Slide 14 / 19

    Calibrate against what chance produces

    • p = 0.003 on its own, and 0.11 after correcting for having looked at thirty-eight facilities — Three facilities fall…
    Speaker notes
    p = 0.003 on its own, and 0.11 after correcting for having looked at thirty-eight facilities. Three facilities fall below p = 0.05 uncorrected, and 1.9 are expected by chance. Three against 1.9 is nothing. There is a second reason to let it go. FAC020's monthly doses run from five to thirteen, so "ends in zero" almost always means the value was exactly ten. On counts that small, the check has no power to distinguish rounding from arithmetic.
  15. Slide 15 / 19

    Calibrate against what chance produces

    The result is: no evidence of digit preference in the vaccination reporting. That is a real finding, it took four lines, and it is the one you should hope for.
  16. Slide 16 / 19

    Why the null case matters more

    • Compute the base rate from the data, not from theory. Terminal digits are not uniform when counts are small, when a…
    • Correct for the number of units you looked at. Testing thirty-eight facilities at 5% produces about two positives…
    • Require a mechanism. Team 2's 69% has one — the team read the measuring board to the nearest half centimetre. If…
    Speaker notes
    Most of what makes this check dangerous is that it is almost always run on data where nothing is wrong, by someone hoping to find something. Three rules that keep it honest.
  17. Slide 17 / 19

    What you may write

    Do not writeWrite
    "FAC020's data appear fabricated""FAC020 reports round numbers more often than the district average; the difference is not significant after adjusting for the number of facilities examined"
    "Team 2's measurements are unreliable""Team 2's height readings end in .0 or .5 in 69% of cases against 18–23% for other teams, consistent with reading the board to the nearest half centimetre"
    "Ages are inaccurate""24% of ages fall on an exact whole year against 9% expected, indicating age was estimated rather than documented for a substantial share of children"
    Speaker notes
    The wording is not a courtesy; it decides what happens next. The right-hand column says what was observed, how large it is, and what mechanism it is consistent with. The left-hand column says what somebody did. Only one of the two produces a corrective action, and only one of them survives being shown to the team it describes.
  18. Slide 18 / 19

    What comes next

    • You now have findings — a completeness collapse, a verification factor of 1.42, a team rounding its heights.
    Speaker notes
    You now have findings — a completeness collapse, a verification factor of 1.42, a team rounding its heights. None of them is yet a cause, and none of them can be acted on. The next lesson is how to get from a defect to the thing that produced it, and why "the staff need training" is almost always the wrong answer.
  19. Slide 19 / 19

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson