76.3% flagged by one, 3.4% by all four

Four answers to one question

Every instrument in the last two lessons claims to identify food-insecure households. Put them on the same households and count.

import pandas as pd

survey = pd.read_csv("food-security-survey-2024.v1.csv")
coping = pd.read_csv("livelihood-coping-2024.v1.csv")
data = survey.merge(coping, on="household_id", how="inner", suffixes=("", "_lcs"))

flags = pd.DataFrame({
    "poor_consumption": fcs <= 28,          # FCS on the 28/42 set
    "hunger": hhs >= 2,                     # HHS moderate or severe
    "high_coping": rcsi >= 19,              # rCSI at or above the median
    "crisis_strategies": severity.isin(["crisis", "emergency"]),
})
flags = flags.dropna()
print(f"analysable on all four: {len(flags)} of {len(survey)}")
print((flags.mean() * 100).round(1))

library(dplyr)

flags |> summarise(across(everything(), ~ mean(.x)), n = n())

Instrument	Households flagged	Share
Poor food consumption	144	7.4%
Moderate or severe hunger	839	42.9%
High coping index	997	51.0%
Crisis or emergency strategies	943	48.2%

A sevenfold range on the same 1,955 households. These are not four estimates of one quantity with sampling error between them. They are four different quantities, each correctly measured.

The overlap

count = flags.sum(axis=1)
print(count.value_counts().sort_index())
print(f"any: {(count >= 1).mean():.1%}   all four: {(count == 4).mean():.1%}")

flags |> mutate(n_flags = rowSums(across(everything()))) |> count(n_flags)

Flagged by	Households	Share
None of the four	464	23.7%
One	591	30.2%
Two	434	22.2%
Three	400	20.5%
All four	66	3.4%

76.3% of households are flagged by at least one instrument and 3.4% by all four. Both numbers are defensible answers to “how many households are food insecure”, and they are twenty-two times apart.

This is the single most important table in the course. Everything that follows — the evidence table, the convergence rule, the working group — exists because this table looks the way it does.

Why they disagree, in order of importance

They measure different moments in the same process. Coping runs ahead of consumption. A household sells its goats in June and is still eating in July, so the LCS flags it and the FCS does not. Reading only consumption sees the crisis about four months late.

They have different sensitivities. The FCS at 7.4% is the least sensitive instrument here by a wide margin, because its threshold was written to identify severe dietary deprivation rather than stress. That is not a defect — it is what makes a poor FCS a strong signal when it appears.

Some ask about behaviour and some about experience. rCSI and LCS ask what a household did; HHS asks what it went through. A household with resources copes without hunger; a household without resources goes hungry without coping, because there is nothing left to sell.

nothing_left = flags["hunger"] & ~flags["crisis_strategies"]
print(f"hungry, no crisis strategies: {nothing_left.sum()} households")

flags |> filter(hunger, !crisis_strategies) |> nrow()

282 households are hungry and using no crisis or emergency strategy. That is the group a coping-only analysis loses, and it is the group furthest along.

What not to do about it

Three repairs that all look principled.

Average them into a composite score. There is no defensible weighting, the instruments are on different scales, and a household at 7.4% on one and 51% on another is not “at 29%”. Composite food security indices exist and every one of them buries the disagreement rather than resolving it.

Pick the one that gives the number you expected. This happens more often than the previous one and is harder to see, because each individual choice is defensible in isolation. Choose the instrument before you compute it, on the question being asked, and write the choice down.

Take the intersection to be safe. 3.4% is the most conservative estimate and it excludes the 400 households flagged by three instruments out of four. Being conservative about a caseload is not caution; it is a decision to serve fewer people, and it should be made as one.

What to do instead

Report all four, with what each is for.

Food security indicators, lean season 2024, 1,955 households

  Poor food consumption (FCS ≤28)          7.4%   severe dietary deprivation
  Moderate or severe hunger (HHS ≥2)      42.9%   experienced deprivation
  High coping (rCSI ≥19)                  51.0%   consumption-based coping
  Crisis or emergency strategies (LCS)    48.2%   asset depletion

  Flagged by at least one                 76.3%
  Flagged by all four                      3.4%

  These are four different quantities, not four estimates of one. The
  spread between them is the finding: coping and asset depletion are
  widespread while severe dietary deprivation is not yet, which is the
  profile of a population early in a deterioration rather than late in one.

The last sentence is the analysis. The pattern across the four — high coping, high asset depletion, low severe consumption deficit — is a diagnosis, and it is only available because the four disagree. A single composite would have produced one number and no diagnosis at all.

The rule this course is built on

Convergence of evidence means agreement across independent instruments raises confidence, and disagreement is information rather than error.

Where three instruments agree and one does not, ask what the fourth measures that the others do not, before deciding it is wrong. In this population the odd one out is the FCS, and what it measures that the others do not is severity — so the disagreement says the deterioration has not yet reached diets, which is a finding about timing and an argument for acting now.

What comes next

Everything so far is what households reported about themselves. The next unit is the market they buy from — an independent line of evidence that does not depend on anybody’s recall, and that turns out to explain the timing of all four indicators.