cassionData Analysis

Back to the lessonLesson 2 of 8The sample is not the population

Building the weights from the frame

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 31

    What this lesson covers

    • A weight is one number with one meaning
    • Stage one: probability proportional to size
    • Stage two: a fixed take
    • The two probabilities cancel
    • The non-response adjustment
    • The check that proves the whole construction
    • Normalised weights, and when to use them
    • Person-level and sub-sample weights
    • Save the weights beside the data
    • What comes next
    Speaker notes
    Selection probability at each stage, the base weight it implies, the non-response adjustment, and the check that proves the whole thing — weights that sum to 56,428, which is exactly what the frame says exists.

  2. Slide 2 / 31

    A weight is one number with one meaning

    • A sampling weight is how many population units this sampled unit stands for
    Speaker notes
    A sampling weight is how many population units this sampled unit stands for. That sentence is the whole of this lesson. Everything else is arithmetic to get there, and every check is a way of asking whether the weights still mean it. The weight is the reciprocal of the probability of selection. A household with a 1-in-60 chance of being picked represents 60 households; a household with a 1-in-18 chance represents 18. Nothing more mysterious than that.

  3. Slide 3 / 31

    Stage one: probability proportional to size — Example

    P(area i selected) = n_h * M_i / M_h
    Speaker notes
    Twenty-five enumeration areas were drawn from each stratum, with a probability proportional to the households each area holds.

  4. Slide 4 / 31

    Stage one: probability proportional to size — In Python

    import pandas as pd

frame = pd.read_csv("household-survey-frame-2025.v1.csv")

stratum_households = frame.groupby("stratum")["households"].sum()
selected = frame[frame["selected"] == True].copy()

selected["p_stage1"] = (
    25 * selected["households"] / selected["stratum"].map(stratum_households)
)
print(selected[["ea_id", "stratum", "households", "p_stage1"]].head())
    Speaker notes
    where n_h is areas sampled in stratum h (25), M_i is households in area i, and M_h is households in the whole stratum.

  5. Slide 5 / 31

    Stage one: probability proportional to size — In R

    library(dplyr)

stratum_households <- frame |> summarise(M_h = sum(households), .by = stratum)

selected <- frame |>
  filter(selected) |>
  left_join(stratum_households, by = "stratum") |>
  mutate(p_stage1 = 25 * households / M_h)
    Speaker notes
    A larger area is more likely to be selected. That is the point of PPS: it puts the interviews where the people are, which stops a survey spending a third of its budget on hamlets.

  6. Slide 6 / 31

    Stage two: a fixed take — Example

    P(household selected | area selected) = m / M_i
    Speaker notes
    Fourteen households were drawn from each selected area, from that area's listing.

  7. Slide 7 / 31

    Stage two: a fixed take — In Python

    selected["p_stage2"] = 14 / selected["households"]
selected["p_overall"] = selected["p_stage1"] * selected["p_stage2"]
print(selected.groupby("stratum")["p_overall"].describe()[["min", "max"]])
    Speaker notes
    with m = 14.

  8. Slide 8 / 31

    Stage two: a fixed take — In R

    selected <- selected |>
  mutate(p_stage2 = 14 / households,
         p_overall = p_stage1 * p_stage2)

selected |> summarise(min = min(p_overall), max = max(p_overall), .by = stratum)

  9. Slide 9 / 31

    The two probabilities cancel — Example

    P(household) = (n_h * M_i / M_h) * (m / M_i) = n_h * m / M_h
    Speaker notes
    Multiply them out and M_i disappears:

  10. Slide 10 / 31

    The two probabilities cancel

    • Two consequences worth holding — A weight that varies within a stratum in a PPS design means something is wrong —…
    Speaker notes
    Every household in a stratum has the same overall probability, regardless of how big its area is. The design is self-weighting within a stratum. Run the code and the minimum and maximum p_overall are identical within each stratum, which is the arithmetic confirming it. This is not a coincidence or a convenience — it is precisely why PPS is paired with a fixed take, in DHS, in MICS and in SMART. The size measure that decides which areas are visited is cancelled by the size measure that decides how many households are taken. Two consequences worth holding. A weight that varies within a stratum in a PPS design means something is wrong — usually a frame whose size measure is out of date. And the base weight is a property of the stratum, not of the household, so it can be computed in three numbers.

  11. Slide 11 / 31

    The two probabilities cancel — In Python

    base_weight = stratum_households / (25 * 14)
print(base_weight.round(1))

  12. Slide 12 / 31

    The two probabilities cancel — In R

    stratum_households |> mutate(base_weight = M_h / (25 * 14))

  13. Slide 13 / 31

    The two probabilities cancel

    StratumFrame householdsBase weight
    Urban21,27060.8
    Rural accessible28,95882.7
    Rural remote6,20017.7
    Speaker notes
    One rural remote household stands for eighteen; one rural accessible household stands for eighty-three. That ratio is the entire difference between 32.8% and 29.1%.

  14. Slide 14 / 31

    The non-response adjustment — In Python

    survey = pd.read_csv("household-survey-2025.v1.csv")

interviewed = selected.set_index("ea_id")["households_interviewed"]

survey["base_weight"] = survey["stratum"].map(base_weight)
survey["nr_adjustment"] = 14 / survey["ea_id"].map(interviewed)
survey["weight"] = survey["base_weight"] * survey["nr_adjustment"]

print(survey["weight"].describe()[["min", "max"]].round(1))
    Speaker notes
    Fourteen households were selected per area. Fewer were interviewed — 996 of the 1,050 selected. The households that were interviewed have to carry the ones that were not.

  15. Slide 15 / 31

    The non-response adjustment — In R

    survey <- survey |>
  left_join(select(selected, ea_id, households_interviewed), by = "ea_id") |>
  left_join(mutate(stratum_households, base_weight = M_h / (25 * 14)), by = "stratum") |>
  mutate(weight = base_weight * 14 / households_interviewed)

range(survey$weight)

  16. Slide 16 / 31

    The non-response adjustment

    • Adjust within the area, not within the stratum — The households that did not answer in a given area resemble the…
    Speaker notes
    Weights now run from 17.7 to 96.5. Adjust within the area, not within the stratum. The households that did not answer in a given area resemble the households that did answer in that area far more than they resemble the stratum average. Adjusting at stratum level is easier and throws away exactly the information that makes the adjustment worth making. The assumption is stated and it is not free: non-respondents are assumed to resemble respondents in the same area. Where you have reason to think otherwise — locked compounds in one neighbourhood, a security incident on one day — say so in the limitations, because no weight can fix it.

  17. Slide 17 / 31

    The check that proves the whole construction — In Python

    total = survey["weight"].sum()
frame_total = frame["households"].sum()
print(f"weights sum to {total:,.0f}; frame holds {frame_total:,}")
assert abs(total - frame_total) < 1

  18. Slide 18 / 31

    The check that proves the whole construction — In R

    c(weights = sum(survey$weight), frame = sum(frame$households))

  19. Slide 19 / 31

    The check that proves the whole construction

    • 56,428 and 56,428 — The weights sum to exactly the number of households the frame says exist, which is what "how many…
    Speaker notes
    56,428 and 56,428. The weights sum to exactly the number of households the frame says exist, which is what "how many units this one stands for" has to mean if it means anything. Run this check every time. It catches a stratum whose base weight used the wrong denominator, a non-response adjustment applied twice, and a join that dropped areas — three mistakes that are otherwise invisible because the resulting estimate still looks like a percentage.

  20. Slide 20 / 31

    Normalised weights, and when to use them — In Python

    survey["weight_norm"] = survey["weight"] / survey["weight"].mean()
    Speaker notes
    Some software wants weights averaging one rather than summing to the population.

  21. Slide 21 / 31

    Normalised weights, and when to use them — In R

    survey <- survey |> mutate(weight_norm = weight / mean(weight))
    Speaker notes
    Proportions and means are identical either way — the scaling cancels. Totals are not. A normalised weight cannot estimate "how many households are food insecure", only "what share are". Keep the population-scaled weight as the primary and derive the normalised one where a tool insists.

  22. Slide 22 / 31

    Person-level and sub-sample weights

    • A person-level weight — is the household weight times the household size, because a household of eight represents eight…
    Speaker notes
    Two more weights fall out of the same logic, and both are routinely got wrong. A person-level weight is the household weight times the household size, because a household of eight represents eight times as many people as it does households.

  23. Slide 23 / 31

    Person-level and sub-sample weights — In Python

    people = survey[survey["household_size"].notna()].copy()
people["person_weight"] = people["weight"] * people["household_size"]
print(f"estimated population {people['person_weight'].sum():,.0f}")

  24. Slide 24 / 31

    Person-level and sub-sample weights — In R

    people <- survey |>
  filter(!is.na(household_size)) |>
  mutate(person_weight = weight * household_size)

sum(people$person_weight)

  25. Slide 25 / 31

    Person-level and sub-sample weights

    • A sub-sample weight — applies when one unit is chosen from several
    Speaker notes
    That gives 333,336 against 328,127 in the frame listing — 1.6% apart. Twenty households have no recorded size and drop out, and a frame listing is itself an estimate made months earlier. A gap of that size is normal and worth stating; a gap of 20% means something is wrong. A sub-sample weight applies when one unit is chosen from several. One child under five was measured per household, so a measured child in a household with three eligible children represents three children.

  26. Slide 26 / 31

    Person-level and sub-sample weights — In Python

    children = survey[survey["child_muac_mm"].notna()].copy()
children["child_weight"] = children["weight"] * children["children_under5"]

  27. Slide 27 / 31

    Person-level and sub-sample weights — In R

    children <- survey |>
  filter(!is.na(child_muac_mm)) |>
  mutate(child_weight = weight * children_under5)
    Speaker notes
    Skip that multiplication and children in large households are under-represented — and large households are systematically poorer, so the bias runs in a predictable direction. On this survey, MUAC-based acute malnutrition is 13.3% among the measured children and 11.4% once the child weights are applied.

  28. Slide 28 / 31

    Save the weights beside the data — In Python

    survey[["household_id", "ea_id", "stratum", "base_weight",
        "nr_adjustment", "weight"]].to_csv("outputs/weights.csv", index=False)

  29. Slide 29 / 31

    Save the weights beside the data — In R

    survey |>
  select(household_id, ea_id, stratum, base_weight, households_interviewed, weight) |>
  readr::write_csv(here::here("outputs", "weights.csv"))
    Speaker notes
    Keep the components, not just the final weight. When somebody asks in a year why one household counts for 96 and another for 18, the answer is in the columns rather than in your memory of a script.

  30. Slide 30 / 31

    What comes next

    • You can now produce a population estimate.
    Speaker notes
    You can now produce a population estimate. The next unit asks what it cost: how many households a given precision needs, and why the textbook sample size formula gives an answer that is about half of what a cluster survey actually requires.

  31. Slide 31 / 31

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson