Back to the lesson·Lesson 7 of 8·Reading it against something else
Three answers to one question
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- The meeting
- The three sources
- Hold the measure, vary the selection: 8.7% to 11.4%
- Hold the selection, vary the measure: 11.4% to 14.9%
- What each source is actually good for
- Triangulation, not reconciliation
- The sentence in the report
- When you only have routine data
- What comes next
Speaker notes
Acute malnutrition is 8.7% in the routine screening register, 11.4% in a household survey using the same measure, and 14.9% in a SMART survey using a different one. Decompose the gap into selection and measure, and neither source is wrong.The meeting
- The nutrition cluster has three figures for acute malnutrition in the same population and the same year.
Speaker notes
The nutrition cluster has three figures for acute malnutrition in the same population and the same year. The routine screening register says 8.7%. A household survey says 11.4%. A SMART survey says 14.9%. Somebody will ask which is right. The answer is that all three are, and the useful work is decomposing the gap into its two causes — who was measured and what was measured — because each cause has a different implication for what the programme should do.The three sources — In Python
import pandas as pd muac = pd.read_csv("muac-screening-artibonite-2024.v1.csv") measured = muac[muac["muac_mm"].notna()] routine = ((measured["muac_mm"] < 125) | (measured["oedema"] == True)).mean() print(f"routine screening register: {routine:.1%} on n={len(measured):,}")The three sources — In R
library(dplyr) muac |> filter(!is.na(muac_mm)) |> summarise(rate = mean(muac_mm < 125 | oedema), n = n())The three sources
Source Measure Population Estimate n Routine screening register MUAC Children brought to screening 8.7% 4,146 Household survey MUAC Population sample, weighted 11.4% 648 SMART survey Weight-for-height z Population sample 14.9% 874 Speaker notes
Two things vary across those rows, and they vary one at a time, which is what makes the decomposition possible.Hold the measure, vary the selection: 8.7% to 11.4%
- Routine data measures the served population, not the population — That is not a defect of the system; it is what a…
Speaker notes
Rows one and two both use MUAC. The only difference is who got measured. The routine register measures children someone brought to a screening. The household survey measures children sampled from the population, with the weights the survey course built. That is a 2.7-point gap, and it goes in the direction it always goes. Screening reaches children whose carers can reach a screening point — closer to the site, less constrained by other work, in a household with someone able to travel. The children who are hardest to reach are systematically more likely to be malnourished, and the routine figure does not contain them. Routine data measures the served population, not the population. That is not a defect of the system; it is what a service register is. The mistake is reading it as a prevalence.Hold the selection, vary the measure: 11.4% to 14.9% — In Python
comparison = pd.DataFrame({ "source": ["Routine register", "Household survey", "SMART survey"], "measure": ["MUAC", "MUAC", "weight-for-height z"], "population": ["screened", "population sample", "population sample"], "estimate": [0.087, 0.114, 0.149], }) comparison["vs_previous"] = comparison["estimate"].diff() print(comparison)Speaker notes
Rows two and three are both population samples. The difference is the measurement. MUAC and weight-for-height do not identify the same children. They correlate, but each finds cases the other misses — MUAC is more sensitive to younger and shorter children, weight-for-height to older and taller ones. Applying the standard cut-offs to the same population gives different prevalences, and in most settings weight-for-height gives the higher figure. A 3.5-point gap here, and it means the two figures cannot be read against the same threshold. The 15% emergency threshold for global acute malnutrition is defined against a specific measure, and a MUAC-based prevalence compared to it is comparing two different quantities. The indicator course made this point about naming; here is what it costs.Hold the selection, vary the measure: 11.4% to 14.9% — In R
tibble::tribble( ~source, ~measure, ~population, ~estimate, "Routine register", "MUAC", "screened", 0.087, "Household survey", "MUAC", "population sample", 0.114, "SMART survey", "WHZ", "population sample", 0.149 ) |> mutate(vs_previous = estimate - lag(estimate))Hold the selection, vary the measure: 11.4% to 14.9%
- Selection accounts for 2.7 points. Measure accounts for 3.5 — Between them they account for the whole of the 6.2-point…
Speaker notes
Selection accounts for 2.7 points. Measure accounts for 3.5. Between them they account for the whole of the 6.2-point spread, and neither is an error.What each source is actually good for
Source Good for Not for Routine register Caseload, workload, supply planning, trend within the served population Prevalence, coverage denominators Household survey Population prevalence with an interval, equity across strata Anything monthly; anything about individual facilities SMART survey Prevalence against the IPC and emergency thresholds, comparability with other surveys Anything more often than annually Speaker notes
Read that table and the meeting resolves itself. The programme manager asking "how many children will we admit next quarter" wants the routine register. The cluster asking "has the situation crossed the emergency threshold" wants the SMART survey. Asking either question of the other source produces a defensible-sounding wrong answer.Triangulation, not reconciliation
- A widening gap between routine and survey means screening coverage is falling, or the hardest-to-reach are becoming…
- A routine figure that moves while the survey does not is usually about case finding — a new outreach round, a new…
- A survey figure that moves while routine does not means the programme is not seeing the change. That is the most…
Speaker notes
The instinct when two sources disagree is to reconcile them — to decide which is right and adjust the other. Resist it. The sources are measuring different things and the difference is information. What the difference tells you:Triangulation, not reconciliation — In Python
def gap_over_time(routine_series, survey_series): return pd.DataFrame({ "routine": routine_series, "survey": survey_series, "ratio": survey_series / routine_series, })Triangulation, not reconciliation — In R
gap_over_time <- function(routine, survey) { tibble::tibble(routine, survey, ratio = survey / routine) }Triangulation, not reconciliation
- Track the ratio, not the difference — A ratio of survey to routine is roughly the inverse of screening coverage, and it…
Speaker notes
Track the ratio, not the difference. A ratio of survey to routine is roughly the inverse of screening coverage, and it is stable enough to be a monitoring indicator in its own right.The sentence in the report — Example
Acute malnutrition in the district, 2024: 14.9% (95% CI 12.3-17.9) by weight-for-height z-score, from a 30-cluster SMART survey of 874 children, design effect 1.5. 11.4% by MUAC in the same population, from a household survey of 648 measured children, weighted to the sampling frame. 8.7% by MUAC among 4,146 children brought to routine screening. This is a measure of the screened population, not a prevalence estimate, and is reported here for comparison with previous rounds of screening only. The gap between the first two figures is a measurement difference; between the second and third, a difference in who was measured. Neither indicates an error.Speaker notes
Ten lines, and they end an argument that would otherwise recur every quarter. The last sentence is the one that does the work.When you only have routine data
- Report caseload, not prevalence. "3,700 children screened, 362 acutely malnourished by MUAC" is a true sentence…
- Report the trend, not the level. A routine series is far more reliable about direction than about level, provided…
- Anchor to the last survey. Where a survey exists, the routine-to-survey ratio from that year can be carried forward…
Speaker notes
Which is most of the time. Three honest positions: What you must not do is present a routine screening rate as a population prevalence. It is the most common misuse of routine data in this sector and it always understates.What comes next
- Every lesson in this course has been a version of one question.
Speaker notes
Every lesson in this course has been a version of one question. The last lesson asks it directly — what exactly was counted, by whom, over what period — and turns it into a block you attach to any figure the system produces.