cassionData Analysis

Back to the lessonLesson 2 of 8Compute it before you believe it

The thirty-seven households zero-filling would misclassify

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 21

    What this lesson covers

    • Two more instruments, two more rules
    • The Household Hunger Scale
    • What zero-filling actually costs
    • The reduced Coping Strategy Index
    • Three instruments, three analysable samples
    • What comes next
    Speaker notes
    Zero-filling an incomplete Household Hunger Scale moves the prevalence by three tenths of a point and can misclassify every one of the thirty-seven households it touches. Which of those matters depends on whether your output is a percentage or a list.
  2. Slide 2 / 21

    Two more instruments, two more rules

    InstrumentWhat it asksThe rule that gets broken
    HHSThree questions on going without food, scored 0–2 eachValid only when all three are answered
    rCSIFive behaviours, days in the last seven, weightedThe weights are not all 1, and the fifth is 3
    Speaker notes
    The Household Hunger Scale and the reduced Coping Strategy Index sit beside the Food Consumption Score in almost every food security survey, and each carries a scoring rule that a naive sum() breaks in a different way.
  3. Slide 3 / 21

    The Household Hunger Scale — In Python

    import pandas as pd
    
    survey = pd.read_csv("food-security-survey-2024.v1.csv")
    HHS = ["hhs_no_food_in_house", "hhs_sleep_hungry",
           "hhs_day_and_night_without_eating"]
    
    complete = survey[HHS].notna().all(axis=1)
    score = survey.loc[complete, HHS].sum(axis=1)
    
    bands = pd.cut(score, [-1, 1, 3, 6],
                   labels=["little or none", "moderate", "severe"])
    print(f"complete: {complete.sum()} of {len(survey)}")
    print((bands.value_counts(normalize=True) * 100).round(1))
    Speaker notes
    Three questions, deliberately few, deliberately severe: was there no food of any kind in the house, did anyone go to sleep hungry, did anyone go a whole day and night without eating. Each is scored 0 for never, 1 for rarely or sometimes, 2 for often.
  4. Slide 4 / 21

    The Household Hunger Scale — In R

    library(dplyr)
    
    hhs <- c("hhs_no_food_in_house", "hhs_sleep_hungry",
             "hhs_day_and_night_without_eating")
    
    survey |>
      filter(if_all(all_of(hhs), ~ !is.na(.x))) |>
      mutate(score = rowSums(across(all_of(hhs))),
             band = cut(score, c(-1, 1, 3, 6),
                        labels = c("little or none", "moderate", "severe"))) |>
      count(band) |> mutate(share = n / sum(n))
  5. Slide 5 / 21

    The Household Hunger Scale

    BandHouseholdsShare
    Little or no hunger (0–1)1,18657.2%
    Moderate (2–3)56427.2%
    Severe (4–6)32515.7%
  6. Slide 6 / 21

    The Household Hunger Scale

    • 2,075 of 2,112 households answered all three — The thirty-seven that did not are the interesting ones
    Speaker notes
    2,075 of 2,112 households answered all three. The thirty-seven that did not are the interesting ones.
  7. Slide 7 / 21

    What zero-filling actually costs — In Python

    zero_filled = survey[HHS].fillna(0).sum(axis=1)
    naive = pd.cut(zero_filled, [-1, 1, 3, 6],
                   labels=["little or none", "moderate", "severe"])
    print((naive.value_counts(normalize=True) * 100).round(1))
    Speaker notes
    The instinct is that dropping the incomplete responses loses data, so fill the gap with zero and keep them. Try it and measure.
  8. Slide 8 / 21

    What zero-filling actually costs — In R

    survey |>
      mutate(score = rowSums(across(all_of(hhs)), na.rm = TRUE)) |>
      count(band = cut(score, c(-1, 1, 3, 6)))
  9. Slide 9 / 21

    What zero-filling actually costs

    Complete casesZero-filled
    Little or none57.2%57.5%
    Moderate27.2%26.9%
    Severe15.7%15.5%
  10. Slide 10 / 21

    What zero-filling actually costs

    • Three tenths of a percentage point — On a prevalence, zero-filling is practically harmless here, and if you stop at the…
    Speaker notes
    Three tenths of a percentage point. On a prevalence, zero-filling is practically harmless here, and if you stop at the table you will conclude it does not matter. Now look at the households rather than the percentage.
  11. Slide 11 / 21

    What zero-filling actually costs — In Python

    partial = survey.loc[~complete, HHS]
    print(partial.sum(axis=1).value_counts().sort_index())
  12. Slide 12 / 21

    What zero-filling actually costs — In R

    survey |> filter(if_any(all_of(hhs), is.na)) |>
      mutate(observed = rowSums(across(all_of(hhs)), na.rm = TRUE)) |> count(observed)
  13. Slide 13 / 21

    What zero-filling actually costs

    • So the cost of zero-filling depends entirely on what the analysis produces
    • If the output is a prevalence, the damage is three tenths of a point and you should say you zero-filled and move on.
    • If the output is a targeting list, you have just told thirty-seven households they are food secure on the strength…
    • Name the output before choosing the rule — This is the same decision as the CMAM cure-rate denominator and the water…
    Speaker notes
    Twenty-nine of the thirty-seven score 0 or 1 on the questions they did answer, so zero-filling classifies them as little or no hunger. The unanswered question is worth up to 2 points. A household sitting at 1 with one question missing could be at 1 or at 3 — little hunger or moderate — and nothing in the data decides it. So the cost of zero-filling depends entirely on what the analysis produces. Name the output before choosing the rule. This is the same decision as the CMAM cure-rate denominator and the water point functionality rate: the defensible choice is not a property of the data, it is a property of the decision the number feeds.
  14. Slide 14 / 21

    The reduced Coping Strategy Index — In Python

    RCSI = {
        "rcsi_less_preferred_food": 1,
        "rcsi_borrowed_food": 2,
        "rcsi_limit_portion_size": 1,
        "rcsi_restrict_adult_consumption": 3,
        "rcsi_reduce_meal_numbers": 1,
    }
    rcsi = sum(survey[column] * weight for column, weight in RCSI.items())
    print(f"median {rcsi.median():.0f}, mean {rcsi.mean():.1f}, max {rcsi.max():.0f}")
    print(f"rCSI 19 or above: {(rcsi >= 19).mean():.1%}")
    Speaker notes
    Five behaviours, days in the last seven, and the weights are the whole point.
  15. Slide 15 / 21

    The reduced Coping Strategy Index — In R

    rcsi_weights <- c(rcsi_less_preferred_food = 1, rcsi_borrowed_food = 2,
                      rcsi_limit_portion_size = 1,
                      rcsi_restrict_adult_consumption = 3,
                      rcsi_reduce_meal_numbers = 1)
  16. Slide 16 / 21

    The reduced Coping Strategy Index

    • Median 19, mean 19.5, and 50.9% at or above 19 — The weight of 3 on restricting adult consumption so children can eat…
    • It has no universal threshold — Unlike the FCS, the rCSI's cut-offs are context-specific and usually set against the…
    • It goes up before consumption goes down — and it can also go down in a crisis, when a household has exhausted the…
    Speaker notes
    Median 19, mean 19.5, and 50.9% at or above 19. The weight of 3 on restricting adult consumption so children can eat is not arbitrary — it is the behaviour most predictive of deterioration, and an unweighted sum would bury it among four behaviours weighted 1. Two things about the rCSI that are routinely got wrong. It has no universal threshold. Unlike the FCS, the rCSI's cut-offs are context-specific and usually set against the distribution in the same population. A report quoting "rCSI above 19" as though 19 were a standard has borrowed a number from another country's analysis. It goes up before consumption goes down, and it can also go down in a crisis, when a household has exhausted the strategies. A falling rCSI beside a falling FCS is worse news than a rising one.
  17. Slide 17 / 21

    Three instruments, three analysable samples — In Python

    denominators = pd.DataFrame({
        "instrument": ["Food Consumption Score", "Household Hunger Scale",
                       "reduced Coping Strategy Index"],
        "analysable": [1989, 2075, 2112],
        "excluded": [123, 37, 0],
        "rule": ["all eight groups answered", "all three questions answered",
                 "complete in this file"],
    })
    print(denominators)
  18. Slide 18 / 21

    Three instruments, three analysable samples — In R

    # Print this before the results, every time.
  19. Slide 19 / 21

    Three instruments, three analysable samples

    • Three numbers on three denominators, and none of them is 2,112 — The difference is small enough to be invisible in a…
    Speaker notes
    Three numbers on three denominators, and none of them is 2,112. The difference is small enough to be invisible in a table and large enough to matter when someone subtracts two of your percentages.
  20. Slide 20 / 21

    What comes next

    • Consumption and hunger say what a household is experiencing.
    Speaker notes
    Consumption and hunger say what a household is experiencing. Neither says what it is doing about it, and the next lesson is the module that does — where the scoring rule is not a sum at all.
  21. Slide 21 / 21

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson