cassionData Analysis

Back to the lessonLesson 7 of 8Judging a programme

The denominator that moves the cure rate nine points

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 26

    What this lesson covers

    • Four outcomes and a standard
    • The three candidate denominators
    • Which one Sphere means
    • Break it down by site
    • The cause is usually distance, not compliance
    • Weight gain, and the entries that cannot be right
    • The performance table to publish
    • What comes next
    Speaker notes
    Cured, defaulted, died, non-response — against the Sphere minimum standards, on the denominator Sphere actually specifies. 82.3% or 73.6%, depending on a choice nobody documents, and one site outside the standard the district figure hides.
  2. Slide 2 / 26

    Four outcomes and a standard

    OutcomeSphere minimum, outpatient therapeutic care
    Curedover 75%
    Diedunder 10%
    Defaultedunder 15%
    Non-responseno fixed threshold; investigated if high
    Speaker notes
    A CMAM programme is judged on what happened to the children it admitted, and the Sphere minimum standards give the thresholds.
  3. Slide 3 / 26

    Four outcomes and a standard

    • Every one of those is a proportion, and the argument is always the denominator
    Speaker notes
    Supplementary feeding has its own set — cured over 75%, died under 3%, defaulted under 15%. The numbers differ; the structure does not. Every one of those is a proportion, and the argument is always the denominator.
  4. Slide 4 / 26

    The three candidate denominators — In Python

    import pandas as pd
    
    cmam = pd.read_csv("cmam-admissions-2024.v1.csv")
    
    print(f"admissions:            {len(cmam):>5}")
    print(f"with an outcome:       {cmam['outcome'].notna().sum():>5}")
    print(f"excluding transfers:   "
          f"{(cmam['outcome'].notna() & (cmam['outcome'] != 'transferred')).sum():>5}")
  5. Slide 5 / 26

    The three candidate denominators — In R

    cmam |> summarise(
      admissions = n(),
      with_outcome = sum(!is.na(outcome)),
      excluding_transfers = sum(!is.na(outcome) & outcome != "transferred")
    )
  6. Slide 6 / 26

    The three candidate denominators

    DenominatornWhat it treats the excluded as
    All admissions1,100Children still in treatment counted as not cured
    Reached an outcome1,029Transfers counted as an outcome
    Reached an outcome, excluding transfers984—
  7. Slide 7 / 26

    The three candidate denominators — In Python

    cured = (cmam["outcome"] == "cured").sum()
    for label, n in [("all admissions", 1100), ("with an outcome", 1029),
                     ("excluding transfers", 984)]:
        print(f"cure rate on {label:22} {cured / n:.1%}")
  8. Slide 8 / 26

    The three candidate denominators — In R

    cured <- sum(cmam$outcome == "cured", na.rm = TRUE)
    c(all = cured / 1100, outcome = cured / 1029, sphere = cured / 984)
  9. Slide 9 / 26

    The three candidate denominators

    • 73.6%, 78.7%, 82.3% — One numerator, three denominators, nine points
    Speaker notes
    73.6%, 78.7%, 82.3%. One numerator, three denominators, nine points. And the middle one crosses a threshold: on all admissions the programme is below the Sphere minimum of 75% and on the Sphere denominator it is comfortably above. The difference is not performance. It is a denominator nobody wrote down.
  10. Slide 10 / 26

    Which one Sphere means

    • Children who reached a treatment outcome, excluding transfers — 984 here
    • Children still in treatment are not an outcome — Seventy-one children were admitted late enough that they had not been…
    • Transfers are somebody else's outcome — Forty-five children moved between programmes — outpatient to inpatient, or the…
    Speaker notes
    Children who reached a treatment outcome, excluding transfers. 984 here. The two exclusions are for different reasons and both are right. Children still in treatment are not an outcome. Seventy-one children were admitted late enough that they had not been discharged at the cut-off. They are neither cured nor defaulted; they are in treatment. Counting them as anything is a claim about a future that has not happened. Transfers are somebody else's outcome. Forty-five children moved between programmes — outpatient to inpatient, or the reverse — and their treatment outcome will be recorded where they end up. Counting them here would count them twice across the two programmes, or credit this programme with a result it did not produce.
  11. Slide 11 / 26

    Which one Sphere means — In Python

    performance = cmam[cmam["outcome"].notna() & (cmam["outcome"] != "transferred")]
    
    rates = performance["outcome"].value_counts(normalize=True)
    print((rates * 100).round(1))
    print(f"n = {len(performance)}")
  12. Slide 12 / 26

    Which one Sphere means — In R

    performance <- cmam |> filter(!is.na(outcome), outcome != "transferred")
    
    performance |> count(outcome) |> mutate(rate = n / sum(n))
  13. Slide 13 / 26

    Which one Sphere means

    OutcomeRateSphere
    Cured82.3%over 75%pass
    Defaulted13.7%under 15%pass
    Died1.0%under 10%pass
    Non-response2.9%—
    Speaker notes
    Four rows, all inside the standard. This is where most CMAM reports stop, and it is where the finding is hiding.
  14. Slide 14 / 26

    Break it down by site — In Python

    by_site = (
        performance.assign(**{o: performance["outcome"] == o
                              for o in ["cured", "defaulted", "died", "non-response"]})
        .groupby("site_id")
        .agg(n=("outcome", "size"), cured=("cured", "mean"),
             defaulted=("defaulted", "mean"), died=("died", "mean"))
    )
    print((by_site[["cured", "defaulted", "died"]] * 100).round(1))
  15. Slide 15 / 26

    Break it down by site — In R

    performance |>
      summarise(n = n(),
                cured = mean(outcome == "cured"),
                defaulted = mean(outcome == "defaulted"),
                died = mean(outcome == "died"),
                .by = site_id) |>
      arrange(desc(defaulted))
  16. Slide 16 / 26

    Break it down by site

    SitenDefaulted
    SITE-0315620.5%
    SITE-0216815.5%
    SITE-0612513.6%
    SITE-0514211.3%
    SITE-0118811.2%
    SITE-0420511.2%
  17. Slide 17 / 26

    Break it down by site

    • The programme defaults at 13.7% and one site defaults at 20.5% — The district figure is inside the Sphere maximum and…
    Speaker notes
    The programme defaults at 13.7% and one site defaults at 20.5%. The district figure is inside the Sphere maximum and that site is not, by five points. That is the whole reason the DQA course insisted on distributions over means, and it arrives here with a clinical consequence: a defaulting child is a child who stopped treatment before recovering.
  18. Slide 18 / 26

    Break it down by site — In Python

    flagged = by_site[(by_site["defaulted"] > 0.15) | (by_site["cured"] < 0.75)]
    print(f"{len(flagged)} of {len(by_site)} sites outside a Sphere standard")
  19. Slide 19 / 26

    Break it down by site — In R

    performance |>
      summarise(cured = mean(outcome == "cured"),
                defaulted = mean(outcome == "defaulted"), .by = site_id) |>
      filter(defaulted > 0.15 | cured < 0.75)
    Speaker notes
    Two sites, and the second is marginal at 15.5%. Flag against the standard, not against the district mean — the standard is what the programme is held to.
  20. Slide 20 / 26

    The cause is usually distance, not compliance — In Python

    length_of_stay = (
        pd.to_datetime(performance["discharge_date"])
        - pd.to_datetime(performance["admission_date"])
    ).dt.days
    
    print(performance.assign(stay=length_of_stay)
          .groupby("outcome")["stay"].median().round(0))
    Speaker notes
    Defaulting has a literature and it is consistent: the strongest predictor is travel time to the site. A carer who must lose a day of work every week to walk two hours will stop coming when the child looks better, which is rational and is not non-compliance.
  21. Slide 21 / 26

    The cause is usually distance, not compliance — In R

    performance |>
      mutate(stay = as.integer(as.Date(discharge_date) - as.Date(admission_date))) |>
      summarise(median_stay = median(stay), .by = outcome)
    Speaker notes
    Defaulters leave far earlier than cured children — the median stay is about three weeks against about eight. A defaulter is not a treatment failure; it is a child who left before the treatment finished, and the corrective action is decentralisation or transport support, not counselling. That is the root cause discipline from the DQA course, applied to a clinical indicator.
  22. Slide 22 / 26

    Weight gain, and the entries that cannot be right — In Python

    gain = (
        (performance["weight_discharge_kg"] - performance["weight_admission_kg"])
        / performance["weight_admission_kg"]
        / length_of_stay * 1000
    )
    print(gain.describe().round(1))
    print(f"{(gain < 0).sum()} discharges with weight loss")
  23. Slide 23 / 26

    Weight gain, and the entries that cannot be right — In R

    performance |>
      mutate(gain = (weight_discharge_kg - weight_admission_kg) /
                    weight_admission_kg / as.integer(as.Date(discharge_date) -
                    as.Date(admission_date)) * 1000) |>
      summarise(median = median(gain, na.rm = TRUE), negative = sum(gain < 0, na.rm = TRUE))
    Speaker notes
    Weight gain in grams per kilogram per day is the standard measure of treatment response, and nine discharges show weight loss. That is possible in a child who died or defaulted early, and it is also exactly what a transposed entry looks like. The register cannot distinguish them, which is a finding for the data quality section rather than a number to clean away.
  24. Slide 24 / 26

    The performance table to publish — Example

    CMAM performance, 2024
    
      Admissions                        1,100
      Still in treatment at cut-off        71    excluded: not an outcome
      Transferred                          45    excluded: outcome recorded elsewhere
      Performance denominator             984
    
      Cured           82.3%   Sphere >75%    pass
      Defaulted       13.7%   Sphere <15%    pass
      Died             1.0%   Sphere <10%    pass
      Non-response     2.9%
    
      One site (SITE-03, n=156) defaults at 20.5%, outside the Sphere maximum.
      Median stay for defaulters is 24 days against 58 for cured children,
      consistent with distance rather than non-response to treatment.
    Speaker notes
    The excluded rows are shown, not deleted. A reader can reconstruct any of the three denominators from that block, which is the difference between a performance table and an assertion.
  25. Slide 25 / 26

    What comes next

    • The programme cures the children it admits.
    Speaker notes
    The programme cures the children it admits. The last lesson asks the harder question — what share of the children who needed treatment it ever saw — and why the figure most programmes report as coverage is not coverage.
  26. Slide 26 / 26

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson