Back to the lesson·Lesson 8 of 8·What the comparison can claim
The sentence this analysis is entitled to
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- Four sentences, one set of numbers
- Why the first three fail
- The four things a claim needs
- Strength of evidence, and where routine data sits
- Write the limitation as a decision, not an apology
- The habit this course was for
- What comes next
Speaker notes
Nord's case fatality is 6.30% against Centre's 2.00%, and the delay gradient rests on one death among thirty cases. Four claims could be written from that, three of them are wrong, and the difference between them is the whole of this course.Four sentences, one set of numbers
- "Nord's cholera response is failing: case fatality there is three times Centre's."
- *"Late presentation causes death — cases admitted four or more days after onset die at 3.3% against 1.9% for those…
- *"Case fatality in Nord is 6.30% (24 deaths among 381 cases with a recorded outcome) against 2.00% in Centre (8 of…
- "The data cannot support a comparison between districts."
- The third is the only defensible one, and it is also the only one long enough to be useful — Work out why each of the…
Speaker notes
Everything this course computed points at one comparison, and there are four ways to write it up. The third is the only defensible one, and it is also the only one long enough to be useful. Work out why each of the others fails and you have the content of this lesson.Why the first three fail
- "Nord's response is failing" — attributes a difference to a cause the data does not distinguish
- "Late presentation causes death" — is a stronger claim than the design supports, and it is built on two cells that…
Speaker notes
"Nord's response is failing" attributes a difference to a cause the data does not distinguish. Nord differs from Centre in age structure, water source, distance to treatment, displacement status and register quality. Lesson 6 removed one of those, lesson 7 named the rest and found half of them unmeasured. A comparison with one confounder removed is not a comparison with none. "Late presentation causes death" is a stronger claim than the design supports, and it is built on two cells that cannot carry it. Onset-to-admission delay is measured from a field one district back-fills — 80% of Nord's cases show a same-day delay, which is the admission book talking, not a fast presentation. And the four-or-more-day band that produces the 3.3% contains one death among thirty cases: move that single case and the gradient disappears.Why the first three fail — In Python
import pandas as pd cases = pd.read_csv("cholera-line-list-2024.v1.csv", parse_dates=["onset_date"]) with_onset = cases[cases["onset_date"].notna()] delayed = with_onset.assign(delay=lambda d: ( pd.to_datetime(d["admission_date"]) - d["onset_date"]).dt.days) print(delayed.groupby("district")["delay"] .agg(median="median", same_day=lambda s: (s == 0).mean()).round(3)) band = pd.cut(delayed["delay"], [-1, 1, 3, 99], labels=["0-1", "2-3", "4+"]) print(delayed.dropna(subset=["outcome"]).groupby(band).size())Why the first three fail — In R
library(dplyr) cases |> filter(!is.na(onset_date)) |> mutate(delay = as.numeric(admission_date - onset_date)) |> summarise(median = median(delay), same_day = mean(delay == 0), .by = district)Why the first three fail
- "The data cannot support a comparison" — is the failure mode of the cautious analyst and it is the most expensive of…
Speaker notes
Nord's median delay is nought days with 80% same-day; Centre's and Sud's are two days. And the three bands hold 372, 214 and 30 cases with an outcome. Always print the cell counts under a gradient, because a percentage prints identically whether it rests on three hundred cases or on one. "The data cannot support a comparison" is the failure mode of the cautious analyst and it is the most expensive of the four. A response that stops reporting because its data is imperfect leaves the decision to be made on nothing, and there is no outbreak anywhere with clean data. The job is to say what the data supports, with its limits attached, not to refuse.The four things a claim needs
- The numerator and the denominator, as counts — "6.30%" is unauditable; "24 deaths among 381 cases with a recorded…
- The comparison it is against — A rate with no reference point is a number looking for an opinion
- The adjustment, or its absence — If you standardised, name the standard
- The limitation that could change the direction — Not a paragraph of hedging — the one or two things that, if they went…
Speaker notes
Every one of them is missing from at least one of the first three sentences. The numerator and the denominator, as counts. "6.30%" is unauditable; "24 deaths among 381 cases with a recorded outcome" can be checked and can be added to another district's. It also declares the exclusion — 974 of 975 cases have an outcome, and saying which denominator you used is how a reader knows whether the one still in treatment was counted. The comparison it is against. A rate with no reference point is a number looking for an opinion. Sphere puts cholera case fatality below 1%; 3.90% is four times that and the sentence should say so. The adjustment, or its absence. If you standardised, name the standard. If you did not, say the comparison is crude. The limitation that could change the direction. Not a paragraph of hedging — the one or two things that, if they went the other way, would change what someone should do. Nord's back-filled dates are one. The unmeasured water source is the other.The four things a claim needs — In Python
claim = { "numerator": 24, "denominator": 381, "denominator_definition": "cases with a recorded outcome", "value": "6.30%", "benchmark": "Sphere: below 1%", "adjustment": "none; crude", "limitation": "onset dates back-filled in this district", }Strength of evidence, and where routine data sits
Design What it can claim Where it appears here Routine data, cross-section Association, adjusted for what was measured This whole course Cohort Association with the exposure preceding the outcome The CMAM register, followed forward Case-control Association, efficiently for rare outcomes Investigating the source of an outbreak Randomised trial Causation Almost never in programme M&E Speaker notes
Not all observational findings are equally weak, and the differences have names.Strength of evidence, and where routine data sits
- Programme data lives on the top row — and the top row can carry a great deal of weight if it is honest about which row…
Speaker notes
Programme data lives on the top row, and the top row can carry a great deal of weight if it is honest about which row it is on. What it cannot do is jump to the bottom one because the association is large or the mechanism is plausible. Two of the strongest arguments available to observational work are still open to you, and this outbreak supplies one of each. A finding that survives adjustment for everything you could measure is more credible than one that vanishes. Nord's excess narrowed under age standardisation and did not disappear, which is the stronger of the two results here. A dose–response gradient — fatality rising steadily across delay bands rather than jumping at one cut-point — is harder to explain away than a single contrast. This is the argument that was not available: the top band held thirty cases, and a gradient resting on one death is not a gradient. Neither, taken alone, makes a finding causal. Knowing which of them you have is what tells you how firmly to write.Write the limitation as a decision, not an apology — Example (cont.)
Cholera case fatality, weeks 1-16 Overall 3.90% 38 deaths / 974 cases with an outcome Nord 6.30% 24 / 381 Sud 3.11% 6 / 193 Centre 2.00% 8 / 400 Against the Sphere threshold of 1%, every district is above standard. What this supports: prioritising treatment capacity in Nord. What it does not: attributing the gap to clinical management or to late presentation. Nord's register back-fills onset dates, so its delays are not comparable, and the 4+ day band holds one death among 30 cases. Water source and crowding, the main determinants of cholera severity, are not recorded at all. What would settle it: two weeks of prospectively recorded onset dates inSpeaker notes
The limitations paragraph most reports carry is a list of regrets. Make it a list of consequences instead.Write the limitation as a decision, not an apology — Example (cont.)
Nord, and the WASH assessment already scheduled for week 18.Write the limitation as a decision, not an apology
- "What would settle it" is the line that makes the limitation useful — A limitation with no route out reads as an…
Speaker notes
"What would settle it" is the line that makes the limitation useful. A limitation with no route out reads as an excuse; one with a named next step is a work plan, and it is what turns an imperfect analysis into the first half of a better one.The habit this course was for
- What did I count, over what? Numerator, denominator, and who was excluded.
- What else could explain it? The table from lesson 7, written before the analysis rather than after.
- What should someone do differently because of this? If nothing, the analysis was not worth the time, and if…
Speaker notes
Three sentences, in this order, before writing anything up. Every measure in this course — incidence and prevalence, the cascade, the epidemic curve, attack rates, case fatality, coverage three ways, the standardised rate — is an answer to the first. The last two are what stop the first from being misused, and they are the part no software computes for you.What comes next
- You can defend a rate, adjust a comparison and state what it is entitled to claim.
Speaker notes
You can defend a rate, adjust a comparison and state what it is entitled to claim. The rest of module 4 applies the same discipline in other sectors — WASH service levels, education attendance, protection case management — and Regression for Programme Data comes back to lesson 7's problem with a tool that can hold several confounders at once.