The mean of your sample is not the mean of anything

Two numbers from one file

Take the household survey, count the households classified food insecure, divide by the number of households. That is 32.8%.

Now weight each household by the number of households it represents in the population, and the same file gives 29.1%.

import pandas as pd

survey = pd.read_csv("household-survey-2025.v1.csv")
frame = pd.read_csv("household-survey-frame-2025.v1.csv")

unweighted = (survey["food_insecure"] == "true").mean()
print(f"unweighted: {unweighted:.1%}")

library(dplyr)
library(readr)

survey <- read_csv("household-survey-2025.v1.csv")
frame  <- read_csv("household-survey-frame-2025.v1.csv")

mean(survey$food_insecure == "true")

Both numbers are computed correctly. One of them is an estimate of the population and the other is a description of who happened to be interviewed, and only one of those is what the report claims to be reporting.

Where the gap comes from

Nothing about the arithmetic. Everything about how the sample was drawn.

Stratum	Households in the frame	Areas sampled	Households interviewed	Food insecure
Urban	21,270	25	316	14.6%
Rural accessible	28,958	25	337	36.2%
Rural remote	6,200	25	343	46.4%

Read the first and last columns together. Rural remote is 11% of the population and 34% of the sample, and it has the worst food insecurity by a wide margin.

An unweighted average of the three columns therefore over-represents the worst-off stratum by a factor of three. The 32.8% is not the district’s food insecurity; it is the food insecurity of a population in which a third of households are remote rural, and no such population exists.

by_stratum = (
    survey.assign(insecure=survey["food_insecure"] == "true")
    .groupby("stratum")
    .agg(interviews=("insecure", "size"), rate=("insecure", "mean"))
)
frame_totals = frame.groupby("stratum")["households"].sum()

comparison = by_stratum.join(frame_totals.rename("frame_households"))
comparison["sample_share"] = comparison["interviews"] / comparison["interviews"].sum()
comparison["population_share"] = (
    comparison["frame_households"] / comparison["frame_households"].sum()
)
print(comparison.round(3))

survey |>
  summarise(interviews = n(), rate = mean(food_insecure == "true"), .by = stratum) |>
  left_join(summarise(frame, frame_households = sum(households), .by = stratum),
            by = "stratum") |>
  mutate(sample_share     = interviews / sum(interviews),
         population_share = frame_households / sum(frame_households))

Stratum	Sample share	Population share
Urban	31.7%	37.7%
Rural accessible	33.8%	51.3%
Rural remote	34.4%	11.0%

When those two columns differ, an unweighted estimate is biased, and the size and direction of the bias are entirely predictable from the table.

Why anyone would design it that way

The obvious reaction is that the sample was drawn badly. It was not, and knowing why is most of what this lesson is for.

Equal allocation across unequal strata is a deliberate, standard choice, and it buys something specific: a usable estimate for each stratum separately. Rural remote holds 11% of households, so a sample proportional to population would have put about 110 interviews there — enough for a national figure and nowhere near enough to say anything about the stratum on its own.

The trade is explicit:

• Proportional allocation gives the best national estimate and weak sub-national ones.
• Equal allocation gives comparable sub-national estimates and requires weighting for anything national.

DHS, MICS and most humanitarian assessments choose the second, because the whole point of the survey is usually to compare places. The weights are not a correction for a mistake. They are the price of the design.

The stratum estimates need no weights

A useful thing falls out of the design and it is worth seeing early.

print(by_stratum["rate"].round(3))

survey |> summarise(rate = mean(food_insecure == "true"), .by = stratum)

Within a stratum, this design is self-weighting — every household had the same probability of selection — so the unweighted stratum rate is already the estimate. Weights matter only when you combine strata.

That is why a report can legitimately show unweighted stratum figures beside a weighted total, and why doing so without saying which is which confuses everyone.

The direction is not always the same

Food insecurity falls when you weight. Improved water access rises, and by more.

for outcome in ["food_insecure", "improved_water_source"]:
    print(f"{outcome:24} unweighted {(survey[outcome] == 'true').mean():.1%}")

survey |>
  summarise(across(c(food_insecure, improved_water_source), ~ mean(.x == "true")))

Outcome	Unweighted	Weighted
Food insecure	32.8%	29.1%
Improved water source	62.8%	69.8%

Seven points on the second one. There is no general rule that unweighted estimates are too high or too low: the bias goes in whichever direction the oversampled stratum differs, and it differs by a different amount for every outcome.

So you cannot correct an unweighted figure by rule of thumb, and a report that presents one with a note saying “unweighted, so the true figure is somewhat lower” is guessing.

What to do when there is no frame

Sometimes you inherit a dataset with no frame and no weights. Three honest positions, and none of them is to report the unweighted mean as an estimate.

• Report by stratum only, if the strata are recorded. Sub-national figures are usually what the audience wanted anyway.
• Reconstruct approximate weights from an external population source, and document that they are approximate.
• Say the survey cannot support a population estimate. This is a real answer and it is better than a number that will be quoted for three years.

What comes next

The weights are computable, from the frame this dataset ships. The next lesson builds them from first principles — selection probability at each stage, base weight, non-response adjustment — and then proves them, by checking that they sum to the number of households the frame says exist.