Back to the lesson·Lesson 8 of 8·Judging a programme
Why admissions over expected caseload is not coverage
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- The figure everyone reports
- What it actually measures
- What coverage requires
- The barriers question is the useful half
- What to report when you have no coverage survey
- Where this course leaves you
Speaker notes
The figure most programmes report as coverage is a ratio of two things, neither of which is the number of malnourished children. What coverage actually requires, and what to report when you cannot estimate it.The figure everyone reports — Example
admissions during the period / expected caseload for the periodSpeaker notes
Somewhere in almost every CMAM report is a line like this: and underneath it, if you look, the calculation:The figure everyone reports — Example
population under five x GAM prevalence x incidence correction x coverage targetSpeaker notes
The expected caseload is itself computed, usually as:The figure everyone reports — In Python
UNDER_FIVE = 21500 GAM_PREVALENCE = 0.149 # from the SMART survey INCIDENCE_CORRECTION = 1.6 # new cases arising over the year COVERAGE_TARGET = 0.50 expected = UNDER_FIVE * GAM_PREVALENCE * INCIDENCE_CORRECTION * COVERAGE_TARGET admissions = 1100 print(f"expected caseload {expected:,.0f}; admissions {admissions:,}; " f"'coverage' {admissions / expected:.0%}")The figure everyone reports — In R
expected <- 21500 * 0.149 * 1.6 * 0.50 c(expected = expected, ratio = 1100 / expected)The figure everyone reports
- That number is not coverage — It is worth being precise about what it is, because it is not useless — it is just not…
Speaker notes
That number is not coverage. It is worth being precise about what it is, because it is not useless — it is just not what its label says.What it actually measures
- The numerator is admissions, not children — A child readmitted after relapse is two admissions
- The denominator is a forecast, not a count — Every term in it is an estimate: the population from a census projection,…
- The coverage target is in the denominator — Look at it again
Speaker notes
Coverage is a proportion: children treated, over children who needed treatment. The formula above has neither. The numerator is admissions, not children. A child readmitted after relapse is two admissions. The reach lesson in the indicator course made this general point; here it inflates the numerator by whatever the relapse rate is. The denominator is a forecast, not a count. Every term in it is an estimate: the population from a census projection, the prevalence from a survey with its own interval, the incidence correction from a literature value, and the coverage target from a proposal. Multiply four estimates and the product carries all four uncertainties. The coverage target is in the denominator. Look at it again. A programme that targeted 50% coverage divides by half the caseload, so hitting the target reads as 100%. The metric is constructed to reach 100% when the plan is met, which makes it a plan achievement ratio — a perfectly reasonable management indicator, wearing the wrong name.What it actually measures — In Python
without_target = admissions / (UNDER_FIVE * GAM_PREVALENCE * INCIDENCE_CORRECTION) print(f"ratio without the coverage target in the denominator: {without_target:.0%}")What it actually measures — In R
1100 / (21500 * 0.149 * 1.6)Speaker notes
Take the target out and the same programme reads at half the figure. Nothing about the programme changed.What coverage requires — Example
children with SAM currently in the programme ------------------------------------------------ children with SAM in the populationSpeaker notes
The definition is not in dispute. Coverage is:What coverage requires — In Python
# What a coverage survey produces, and a register never can cases_found = 84 cases_in_programme = 39 print(f"point coverage: {cases_in_programme / cases_found:.0%}")Speaker notes
and the denominator can only come from finding malnourished children in the population who are not in the programme. No register can supply it, because a register only contains children who came. That is why coverage surveys exist as a separate methodology. The current standards — SQUEAC, SLEAC and their successors — do exactly this: active case-finding in the community, then classify each case found as in or out of the programme.What coverage requires
- Point coverage — cases currently in the programme, over cases found. Suits a programme with short treatment…
- Period coverage — cases in the programme or recently discharged cured, over cases found. Suits longer episodes, and…
- State which — They differ by several points and both are called coverage
Speaker notes
Two coverage measures are in use and they answer different questions: State which. They differ by several points and both are called coverage.The barriers question is the useful half
- Did not know the child was malnourished — screening and community awareness.
- Did not know the programme existed — mobilisation.
- Distance, cost, time — decentralisation, which is also the defaulting cause from the previous lesson.
- Rejected previously — an admission criterion problem, and lesson 4 explains how a child can be rejected on one…
- Stigma, or a previous bad experience — quality of care.
Speaker notes
A coverage survey's estimate is a number with a wide interval. Its other output is often more actionable: for every case found and not in the programme, the carer is asked why. The answers cluster, and they map to different fixes:The barriers question is the useful half — In Python
barriers = pd.DataFrame({ "barrier": ["Did not know child was malnourished", "Distance", "Did not know programme existed", "Previously rejected", "Carer unavailable"], "cases": [17, 12, 8, 5, 3], }) barriers["share"] = barriers["cases"] / barriers["cases"].sum()The barriers question is the useful half — In R
barriers <- tibble::tribble( ~barrier, ~cases, "unaware child malnourished", 17, "distance", 12, "unaware of programme", 8, "previously rejected", 5, "carer unavailable", 3 )The barriers question is the useful half
- Report the barriers even when the coverage estimate is too imprecise to use — Forty-five interviews cannot pin a…
Speaker notes
Report the barriers even when the coverage estimate is too imprecise to use. Forty-five interviews cannot pin a coverage figure to five points and can perfectly well tell you that half the missed cases are a screening problem rather than an access problem, and those have different budgets.What to report when you have no coverage survey
- Report the plan achievement ratio, correctly named — "Admissions were 68% of the caseload planned for the period" is…
- Report admissions against expected incident cases, with the assumptions stated — Drop the coverage target from the…
- Use the routine-to-survey ratio — The DHIS2 course's last lesson decomposed the gap between a screening register and a…
Speaker notes
Which is most of the time, because a coverage survey costs money the programme usually does not have. Three honest options, and none is to relabel the ratio. Report the plan achievement ratio, correctly named. "Admissions were 68% of the caseload planned for the period" is true, useful for supply planning, and makes no claim about children who never came. Report admissions against expected incident cases, with the assumptions stated. Drop the coverage target from the denominator, list the four inputs and their sources, and label it an estimate of programme reach against expected need. Use the routine-to-survey ratio. The DHIS2 course's last lesson decomposed the gap between a screening register and a survey; that gap is an indirect signal about who the programme is not seeing, and it is free.What to report when you have no coverage survey — In Python
report = pd.DataFrame({ "indicator": ["Admissions", "Expected incident cases", "Plan achievement", "Coverage"], "value": [1100, 5126, "21%", "not estimated"], "note": ["episodes, not children", "population x prevalence x incidence correction; three estimates", "admissions / expected incident cases", "requires active case-finding in the community"], })What to report when you have no coverage survey — In R
tibble::tribble( ~indicator, ~value, ~note, "Admissions", "1,100", "episodes, not children", "Expected incident cases", "5,126", "three stacked estimates", "Plan achievement", "21%", "admissions / expected", "Coverage", "not estimated", "requires a coverage survey" )Speaker notes
The last row is the one that matters. "Not estimated" is a finding, and it is a far better line in a report than a number that will be quoted for three years as something it is not.Where this course leaves you
- You can measure a child, compute a z-score against the WHO standards, apply the case definitions including the clause everyone drops, say where the two admission criteria disagree and why, run a SMART plausibility report and decide whether a survey can be believed, produce a prevalence with an interval and read it against the IPC phases, compute CMAM performance on the denominator Sphere specifies, and say honestly what your programme does and does not know about coverage.
Speaker notes
You can measure a child, compute a z-score against the WHO standards, apply the case definitions including the clause everyone drops, say where the two admission criteria disagree and why, run a SMART plausibility report and decide whether a survey can be believed, produce a prevalence with an interval and read it against the IPC phases, compute CMAM performance on the denominator Sphere specifies, and say honestly what your programme does and does not know about coverage. That is the nutrition analyst's job, and it is the first of module 4's six sector courses. Next in the module is Public Health and Applied Epidemiology — incidence, prevalence, cascades and coverage for HIV, TB, malaria and immunisation, plus the outbreak curve you may have to draw at short notice. It runs on the vaccination extract this programme has been using since module 2, and it starts where this course's denominators leave off: with person-time, which is the denominator that makes incidence mean anything at all.