cassionData Analysis

Lesson 8 of 8

Unit · What the comparison can claim

The sentence this analysis is entitled to

Nord's case fatality is 6.30% against Centre's 2.00%, and the delay gradient rests on one death among thirty cases. Four claims could be written from that, three of them are wrong, and the difference between them is the whole of this course.

PythonR135 minUNICEF indicator definitionsOECD DAC evaluation criteriaSphere Standards

Four sentences, one set of numbers

Everything this course computed points at one comparison, and there are four ways to write it up.

  1. “Nord’s cholera response is failing: case fatality there is three times Centre’s.”
  2. “Late presentation causes death — cases admitted four or more days after onset die at 3.3% against 1.9% for those admitted sooner.”
  3. “Case fatality in Nord is 6.30% (24 deaths among 381 cases with a recorded outcome) against 2.00% in Centre (8 of 400). Both exceed the 1% Sphere threshold. Onset-to-admission delay does not explain the gap in this data: Nord’s onset dates are back-filled from the admission book, and the highest delay band carries one death among thirty cases.”
  4. “The data cannot support a comparison between districts.”

The third is the only defensible one, and it is also the only one long enough to be useful. Work out why each of the others fails and you have the content of this lesson.

Why the first three fail

“Nord’s response is failing” attributes a difference to a cause the data does not distinguish. Nord differs from Centre in age structure, water source, distance to treatment, displacement status and register quality. Lesson 6 removed one of those, lesson 7 named the rest and found half of them unmeasured. A comparison with one confounder removed is not a comparison with none.

“Late presentation causes death” is a stronger claim than the design supports, and it is built on two cells that cannot carry it. Onset-to-admission delay is measured from a field one district back-fills — 80% of Nord’s cases show a same-day delay, which is the admission book talking, not a fast presentation. And the four-or-more-day band that produces the 3.3% contains one death among thirty cases: move that single case and the gradient disappears.

import pandas as pd

cases = pd.read_csv("cholera-line-list-2024.v1.csv", parse_dates=["onset_date"])
with_onset = cases[cases["onset_date"].notna()]

delayed = with_onset.assign(delay=lambda d: (
    pd.to_datetime(d["admission_date"]) - d["onset_date"]).dt.days)

print(delayed.groupby("district")["delay"]
      .agg(median="median", same_day=lambda s: (s == 0).mean()).round(3))

band = pd.cut(delayed["delay"], [-1, 1, 3, 99], labels=["0-1", "2-3", "4+"])
print(delayed.dropna(subset=["outcome"]).groupby(band).size())
library(dplyr)

cases |>
  filter(!is.na(onset_date)) |>
  mutate(delay = as.numeric(admission_date - onset_date)) |>
  summarise(median = median(delay), same_day = mean(delay == 0), .by = district)

Nord’s median delay is nought days with 80% same-day; Centre’s and Sud’s are two days. And the three bands hold 372, 214 and 30 cases with an outcome. Always print the cell counts under a gradient, because a percentage prints identically whether it rests on three hundred cases or on one.

“The data cannot support a comparison” is the failure mode of the cautious analyst and it is the most expensive of the four. A response that stops reporting because its data is imperfect leaves the decision to be made on nothing, and there is no outbreak anywhere with clean data. The job is to say what the data supports, with its limits attached, not to refuse.

The four things a claim needs

Every one of them is missing from at least one of the first three sentences.

The numerator and the denominator, as counts. “6.30%” is unauditable; “24 deaths among 381 cases with a recorded outcome” can be checked and can be added to another district’s. It also declares the exclusion — 974 of 975 cases have an outcome, and saying which denominator you used is how a reader knows whether the one still in treatment was counted.

The comparison it is against. A rate with no reference point is a number looking for an opinion. Sphere puts cholera case fatality below 1%; 3.90% is four times that and the sentence should say so.

The adjustment, or its absence. If you standardised, name the standard. If you did not, say the comparison is crude.

The limitation that could change the direction. Not a paragraph of hedging — the one or two things that, if they went the other way, would change what someone should do. Nord’s back-filled dates are one. The unmeasured water source is the other.

claim = {
    "numerator": 24,
    "denominator": 381,
    "denominator_definition": "cases with a recorded outcome",
    "value": "6.30%",
    "benchmark": "Sphere: below 1%",
    "adjustment": "none; crude",
    "limitation": "onset dates back-filled in this district",
}
# The same fields, in the footnote of the table.

Strength of evidence, and where routine data sits

Not all observational findings are equally weak, and the differences have names.

Design What it can claim Where it appears here
Routine data, cross-section Association, adjusted for what was measured This whole course
Cohort Association with the exposure preceding the outcome The CMAM register, followed forward
Case-control Association, efficiently for rare outcomes Investigating the source of an outbreak
Randomised trial Causation Almost never in programme M&E

Programme data lives on the top row, and the top row can carry a great deal of weight if it is honest about which row it is on. What it cannot do is jump to the bottom one because the association is large or the mechanism is plausible.

Two of the strongest arguments available to observational work are still open to you, and this outbreak supplies one of each.

A finding that survives adjustment for everything you could measure is more credible than one that vanishes. Nord’s excess narrowed under age standardisation and did not disappear, which is the stronger of the two results here.

A dose–response gradient — fatality rising steadily across delay bands rather than jumping at one cut-point — is harder to explain away than a single contrast. This is the argument that was not available: the top band held thirty cases, and a gradient resting on one death is not a gradient.

Neither, taken alone, makes a finding causal. Knowing which of them you have is what tells you how firmly to write.

Write the limitation as a decision, not an apology

The limitations paragraph most reports carry is a list of regrets. Make it a list of consequences instead.

Cholera case fatality, weeks 1-16

  Overall     3.90%   38 deaths / 974 cases with an outcome
  Nord        6.30%   24 / 381
  Sud         3.11%    6 / 193
  Centre      2.00%    8 / 400

  Against the Sphere threshold of 1%, every district is above standard.

  What this supports: prioritising treatment capacity in Nord.
  What it does not: attributing the gap to clinical management or to late
  presentation. Nord's register back-fills onset dates, so its delays are not
  comparable, and the 4+ day band holds one death among 30 cases. Water
  source and crowding, the main determinants of cholera severity, are not
  recorded at all.
  What would settle it: two weeks of prospectively recorded onset dates in
  Nord, and the WASH assessment already scheduled for week 18.

“What would settle it” is the line that makes the limitation useful. A limitation with no route out reads as an excuse; one with a named next step is a work plan, and it is what turns an imperfect analysis into the first half of a better one.

The habit this course was for

Three sentences, in this order, before writing anything up.

  1. What did I count, over what? Numerator, denominator, and who was excluded.
  2. What else could explain it? The table from lesson 7, written before the analysis rather than after.
  3. What should someone do differently because of this? If nothing, the analysis was not worth the time, and if something, the claim has to be strong enough to bear it.

Every measure in this course — incidence and prevalence, the cascade, the epidemic curve, attack rates, case fatality, coverage three ways, the standardised rate — is an answer to the first. The last two are what stop the first from being misused, and they are the part no software computes for you.

What comes next

You can defend a rate, adjust a comparison and state what it is entitled to claim. The rest of module 4 applies the same discipline in other sectors — WASH service levels, education attendance, protection case management — and Regression for Programme Data comes back to lesson 7’s problem with a tool that can hold several confounders at once.

Teach this lesson

The lesson as a slide deck, with the prose kept in the speaker notes rather than on the slide. Generated from this page, so it cannot fall out of step with it.

Start the slideshowRead the slides

The PDF needs no software and projects from any machine. The PowerPoint file is there to be edited — add your organisation's branding, cut a section for a shorter session, or merge two lessons into a workshop.