cassionData Analysis

Lesson 7 of 8

Unit · Judging a programme

The denominator that moves the cure rate nine points

Cured, defaulted, died, non-response — against the Sphere minimum standards, on the denominator Sphere actually specifies. 82.3% or 73.6%, depending on a choice nobody documents, and one site outside the standard the district figure hides.

PythonR150 minSphere StandardsUNICEF indicator definitionsCore Humanitarian Standard (CHS)

Four outcomes and a standard

A CMAM programme is judged on what happened to the children it admitted, and the Sphere minimum standards give the thresholds.

Outcome Sphere minimum, outpatient therapeutic care
Cured over 75%
Died under 10%
Defaulted under 15%
Non-response no fixed threshold; investigated if high

Supplementary feeding has its own set — cured over 75%, died under 3%, defaulted under 15%. The numbers differ; the structure does not.

Every one of those is a proportion, and the argument is always the denominator.

The three candidate denominators

import pandas as pd

cmam = pd.read_csv("cmam-admissions-2024.v1.csv")

print(f"admissions:            {len(cmam):>5}")
print(f"with an outcome:       {cmam['outcome'].notna().sum():>5}")
print(f"excluding transfers:   "
      f"{(cmam['outcome'].notna() & (cmam['outcome'] != 'transferred')).sum():>5}")
cmam |> summarise(
  admissions = n(),
  with_outcome = sum(!is.na(outcome)),
  excluding_transfers = sum(!is.na(outcome) & outcome != "transferred")
)
Denominator n What it treats the excluded as
All admissions 1,100 Children still in treatment counted as not cured
Reached an outcome 1,029 Transfers counted as an outcome
Reached an outcome, excluding transfers 984 —
cured = (cmam["outcome"] == "cured").sum()
for label, n in [("all admissions", 1100), ("with an outcome", 1029),
                 ("excluding transfers", 984)]:
    print(f"cure rate on {label:22} {cured / n:.1%}")
cured <- sum(cmam$outcome == "cured", na.rm = TRUE)
c(all = cured / 1100, outcome = cured / 1029, sphere = cured / 984)

73.6%, 78.7%, 82.3%. One numerator, three denominators, nine points.

And the middle one crosses a threshold: on all admissions the programme is below the Sphere minimum of 75% and on the Sphere denominator it is comfortably above. The difference is not performance. It is a denominator nobody wrote down.

Which one Sphere means

Children who reached a treatment outcome, excluding transfers. 984 here.

The two exclusions are for different reasons and both are right.

Children still in treatment are not an outcome. Seventy-one children were admitted late enough that they had not been discharged at the cut-off. They are neither cured nor defaulted; they are in treatment. Counting them as anything is a claim about a future that has not happened.

Transfers are somebody else’s outcome. Forty-five children moved between programmes — outpatient to inpatient, or the reverse — and their treatment outcome will be recorded where they end up. Counting them here would count them twice across the two programmes, or credit this programme with a result it did not produce.

performance = cmam[cmam["outcome"].notna() & (cmam["outcome"] != "transferred")]

rates = performance["outcome"].value_counts(normalize=True)
print((rates * 100).round(1))
print(f"n = {len(performance)}")
performance <- cmam |> filter(!is.na(outcome), outcome != "transferred")

performance |> count(outcome) |> mutate(rate = n / sum(n))
Outcome Rate Sphere
Cured 82.3% over 75% pass
Defaulted 13.7% under 15% pass
Died 1.0% under 10% pass
Non-response 2.9% —

Four rows, all inside the standard. This is where most CMAM reports stop, and it is where the finding is hiding.

Break it down by site

by_site = (
    performance.assign(**{o: performance["outcome"] == o
                          for o in ["cured", "defaulted", "died", "non-response"]})
    .groupby("site_id")
    .agg(n=("outcome", "size"), cured=("cured", "mean"),
         defaulted=("defaulted", "mean"), died=("died", "mean"))
)
print((by_site[["cured", "defaulted", "died"]] * 100).round(1))
performance |>
  summarise(n = n(),
            cured = mean(outcome == "cured"),
            defaulted = mean(outcome == "defaulted"),
            died = mean(outcome == "died"),
            .by = site_id) |>
  arrange(desc(defaulted))
Site n Defaulted
SITE-03 156 20.5%
SITE-02 168 15.5%
SITE-06 125 13.6%
SITE-05 142 11.3%
SITE-01 188 11.2%
SITE-04 205 11.2%

The programme defaults at 13.7% and one site defaults at 20.5%. The district figure is inside the Sphere maximum and that site is not, by five points.

That is the whole reason the DQA course insisted on distributions over means, and it arrives here with a clinical consequence: a defaulting child is a child who stopped treatment before recovering.

flagged = by_site[(by_site["defaulted"] > 0.15) | (by_site["cured"] < 0.75)]
print(f"{len(flagged)} of {len(by_site)} sites outside a Sphere standard")
performance |>
  summarise(cured = mean(outcome == "cured"),
            defaulted = mean(outcome == "defaulted"), .by = site_id) |>
  filter(defaulted > 0.15 | cured < 0.75)

Two sites, and the second is marginal at 15.5%. Flag against the standard, not against the district mean — the standard is what the programme is held to.

The cause is usually distance, not compliance

Defaulting has a literature and it is consistent: the strongest predictor is travel time to the site. A carer who must lose a day of work every week to walk two hours will stop coming when the child looks better, which is rational and is not non-compliance.

length_of_stay = (
    pd.to_datetime(performance["discharge_date"])
    - pd.to_datetime(performance["admission_date"])
).dt.days

print(performance.assign(stay=length_of_stay)
      .groupby("outcome")["stay"].median().round(0))
performance |>
  mutate(stay = as.integer(as.Date(discharge_date) - as.Date(admission_date))) |>
  summarise(median_stay = median(stay), .by = outcome)

Defaulters leave far earlier than cured children — the median stay is about three weeks against about eight. A defaulter is not a treatment failure; it is a child who left before the treatment finished, and the corrective action is decentralisation or transport support, not counselling.

That is the root cause discipline from the DQA course, applied to a clinical indicator.

Weight gain, and the entries that cannot be right

gain = (
    (performance["weight_discharge_kg"] - performance["weight_admission_kg"])
    / performance["weight_admission_kg"]
    / length_of_stay * 1000
)
print(gain.describe().round(1))
print(f"{(gain < 0).sum()} discharges with weight loss")
performance |>
  mutate(gain = (weight_discharge_kg - weight_admission_kg) /
                weight_admission_kg / as.integer(as.Date(discharge_date) -
                as.Date(admission_date)) * 1000) |>
  summarise(median = median(gain, na.rm = TRUE), negative = sum(gain < 0, na.rm = TRUE))

Weight gain in grams per kilogram per day is the standard measure of treatment response, and nine discharges show weight loss. That is possible in a child who died or defaulted early, and it is also exactly what a transposed entry looks like. The register cannot distinguish them, which is a finding for the data quality section rather than a number to clean away.

The performance table to publish

CMAM performance, 2024

  Admissions                        1,100
  Still in treatment at cut-off        71    excluded: not an outcome
  Transferred                          45    excluded: outcome recorded elsewhere
  Performance denominator             984

  Cured           82.3%   Sphere >75%    pass
  Defaulted       13.7%   Sphere <15%    pass
  Died             1.0%   Sphere <10%    pass
  Non-response     2.9%

  One site (SITE-03, n=156) defaults at 20.5%, outside the Sphere maximum.
  Median stay for defaulters is 24 days against 58 for cured children,
  consistent with distance rather than non-response to treatment.

The excluded rows are shown, not deleted. A reader can reconstruct any of the three denominators from that block, which is the difference between a performance table and an assertion.

What comes next

The programme cures the children it admits. The last lesson asks the harder question — what share of the children who needed treatment it ever saw — and why the figure most programmes report as coverage is not coverage.

Teach this lesson

The lesson as a slide deck, with the prose kept in the speaker notes rather than on the slide. Generated from this page, so it cannot fall out of step with it.

Start the slideshowRead the slides

The PDF needs no software and projects from any machine. The PowerPoint file is there to be edited — add your organisation's branding, cut a section for a shorter session, or merge two lessons into a workshop.