Back to the lesson·Lesson 4 of 8·The reference sheet
The denominator argument, and the cuts you promise
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- Every argument about a number is an argument about the denominator
- Three denominators, one register
- The rule
- Say what it excludes, in the sheet
- Disaggregation is a promise about sample size
- Set a minimum cell size, in the sheet, in advance
- Which cuts actually change a decision
- What comes next
Speaker notes
Three denominators for one numerator, the rule that decides between them, and the disaggregation that turns twenty-four cells into forty-eight — three of which are too small to report.Every argument about a number is an argument about the denominator
- The numerator is usually agreed within a minute.
Speaker notes
The numerator is usually agreed within a minute. Somebody counted the children with a MUAC below 125 mm and got 362, and nobody disputes it. What follows is an hour about what to divide it by, and the reason the hour happens is that all the candidates are defensible.Three denominators, one register — In Python
measured = muac["muac_mm"].notna() assessed = measured | muac["oedema"].notna() cases = (muac["muac_mm"] < 125) | (muac["oedema"] == True) for label, mask in [("all screened", pd.Series(True, index=muac.index)), ("with a measurement", measured), ("assessed either way", assessed)]: print(f"{label:22} n={mask.sum():5} GAM={cases.sum() / mask.sum():.2%}")Three denominators, one register — In R
muac |> summarise( all_screened = n(), measured = sum(!is.na(muac_mm)), assessed = sum(!is.na(muac_mm) | !is.na(oedema)), cases = sum(muac_mm < 125 | oedema, na.rm = TRUE) ) |> mutate(across(c(all_screened, measured, assessed), ~ cases / .x, .names = "gam_{.col}"))Three denominators, one register
Denominator n GAM Everyone who came 4,218 8.58% Children with a MUAC measurement 4,146 8.73% Children assessed by measurement or oedema 4,216 8.59% Speaker notes
Three numbers, 0.15 points apart, and all three are in use in real reports. The gap is small here, which is exactly why the lesson is worth learning on this file rather than on one where the answer is obvious. The first denominator is wrong, and it is wrong in a way that does not show up as a large difference. Dividing by everyone who came treats the 72 unmeasured children as though they had been measured and found well nourished. That is a claim about children nobody looked at.The rule
The denominator is the population that was genuinely at risk of being in the numerator, over the same period, in the same place.
The rule
- Coverage. Not everyone in the district — the target age group in the catchment, over the period.
- Cure rate. Not everyone admitted — those who reached an outcome. Children still in treatment at the cut-off are…
- Referral completion. Not all cases — cases that consented to be referred, which the protection dataset makes…
- Attendance. Not enrolled children times all school days — enrolled children times the days their school was open.
Speaker notes
Apply it and the answer here falls out. A child with no measurement and no oedema assessment could not have entered the numerator whatever their nutritional status, so they do not belong in the denominator. The second row is the defensible one, and the third is defensible if you count an oedema assessment as sufficient. The same rule settles most of the arguments you will have:Say what it excludes, in the sheet — Example
Denominator Children with a MUAC measurement recorded (n = 4,146 of 4,218). Excludes 72 children (1.7%) screened but not measured, coded -99. Their nutritional status is unknown; if they were systematically the most distressed children, GAM is understated. Not testable from this register.Speaker notes
Whichever you pick, the exclusion is a claim and it goes in writing. That last sentence is the one that makes the number defensible. You have named a direction of possible bias and said you cannot resolve it, which is a stronger position than any figure presented without it.Disaggregation is a promise about sample size — In Python
banded = muac[muac["age_months"].notna()].assign( band=lambda d: (d["age_months"] >= 24).map({True: "24-59", False: "6-23"}) ) by_two = banded.groupby(["commune", "band"]).size() by_three = banded.groupby(["commune", "band", "sex"]).size() print(f"commune x band: {len(by_two)} cells, smallest {by_two.min()}") print(f"commune x band x sex: {len(by_three)} cells, smallest {by_three.min()}, " f"{(by_three < 30).sum()} below 30")Speaker notes
A LogFrame that says "disaggregated by sex, age and district" has committed to producing cells, and cells have counts.Disaggregation is a promise about sample size — In R
banded <- muac |> filter(!is.na(age_months)) |> mutate(band = if_else(age_months >= 24, "24-59", "6-23")) banded |> count(commune, band) |> summarise(cells = n(), smallest = min(n)) banded |> count(commune, band, sex) |> summarise(cells = n(), smallest = min(n), under_30 = sum(n < 30))Disaggregation is a promise about sample size
Disaggregation Cells Smallest cell Cells under 30 Commune × age band 24 49 0 Commune × age band × sex 48 16 3 Disaggregation is a promise about sample size
- Adding one binary cut doubles the cells and halves their size — Going from two dimensions to three takes the smallest…
Speaker notes
Adding one binary cut doubles the cells and halves their size. Going from two dimensions to three takes the smallest cell from 49 children to 16, and puts three cells below any reasonable reporting threshold. A prevalence computed on 16 children moves by six percentage points when one child changes category. Publishing it in a table alongside a commune-level figure computed on 400 invites a reader to compare them as though they were the same kind of number.Set a minimum cell size, in the sheet, in advance — In Python
MIN_CELL = 30 table = ( banded.assign(case=cases) .groupby(["commune", "band", "sex"]) .agg(n=("case", "size"), cases=("case", "sum")) ) table["rate"] = (table["cases"] / table["n"]).where(table["n"] >= MIN_CELL) table["note"] = table["n"].lt(MIN_CELL).map({True: "suppressed: n < 30", False: ""})Set a minimum cell size, in the sheet, in advance — In R
MIN_CELL <- 30 table <- banded |> summarise(n = n(), cases = sum(case), .by = c(commune, band, sex)) |> mutate(rate = if_else(n >= MIN_CELL, cases / n, NA_real_), note = if_else(n < MIN_CELL, "suppressed: n < 30", ""))Speaker notes
Two properties matter. The countnstays visible even where the rate is suppressed, so a reader can see the cell exists and why it is empty. And the threshold was set before the numbers were seen, which is the same discipline the DQA course applied to tolerance bands. In protection and GBV data this stops being a statistical nicety and becomes a disclosure control, which the Protection and GBV Data course covers properly. The habit is the same and it is worth having before you need it.Which cuts actually change a decision
- Sex — yes, almost always. It changes targeting, staffing and messaging.
- Age band — yes for nutrition, where admission criteria differ by age.
- District — yes, it moves resources.
- Facility type — sometimes; it changes supervision, which is real.
- Month — usually a trend, not a disaggregation, and cheaper as a chart.
Speaker notes
The last question, and it is a budget question as much as an analytical one. Every disaggregation you promise has to be collected, cleaned, computed and checked, and most LogFrames promise more than anyone uses. Ask of each cut: would the programme do something different if this cut showed a gap? A cut that would change nothing is a reporting cost with a data quality risk attached, because every additional required field is another field that arrives empty.What comes next
- You can now define one indicator precisely.
Speaker notes
You can now define one indicator precisely. The next unit is about the family of indicators that count people, where the definitional problem is different: the same person appearing in twelve monthly reports, and what happens when somebody adds those reports up.