Course · Intermediate · Indicators and Measurement
Survey Analysis, Sampling and Weighting
Analyse a two-stage cluster survey properly — weights, strata, design effect and confidence intervals — using the conventions DHS, MICS and SMART already impose.
What you will be able to do
- Reconstruct sampling weights from a frame, including the non-response adjustment, and prove they sum to the population
- Say why an unweighted mean from a cluster survey is the wrong number, and how far wrong it is on a given survey
- Compute a sample size for a prevalence estimate, and state what precision the budget actually bought
- Estimate a proportion with a design-adjusted confidence interval in both R and Python, and read the design effect off the result
- Handle non-response and replacement clusters explicitly rather than by ignoring them
- Refuse to disaggregate past the point the sample supports, and say in a report where that point is
Standards and methodologies
A survey estimate is not a number. It is a number, an interval, and a design that produced both — and dropping either of the last two is the most common way survey findings are misreported in this sector.
This is the heaviest course in the programme so far, and the first that genuinely needs statistics. It earns that weight because the alternative is expensive: a household survey costs vehicles, enumerators and six weeks, and analysing it as though it were a spreadsheet of independent observations throws away most of what was bought.
Almost everything runs against one dataset built for this course — 996 household interviews drawn from a 470-area frame by stratified sampling with probability proportional to size. It ships the frame rather than the weights, in the same way the SMART survey ships raw measurements rather than z-scores, because reconstructing the weights is the exercise.
Three findings from that survey shape the course. Food insecurity is 32.8% unweighted and 29.1% weighted, because the smallest stratum is the most oversampled and the worst off. The design effect for that estimate is 1.60, so the survey’s 996 interviews are worth about 623 independent ones. And the design effect for improved water access is 0.91 — below one, because the stratification gain outweighs the clustering loss — which is the result that stops “design effect” being memorised as “a penalty”.
Both Python and R, throughout. R’s survey package is the reference
implementation this sector uses and the course teaches it; the Python examples
compute the same quantities directly, so the arithmetic is visible rather than
delegated.
You should have done Indicator Design and the LogFrame, or at least be in the habit of writing a denominator down before computing anything. This course adds the interval to the number that course taught you to define.
Practice
Reading the lessons is not the same as having done the work. Each of these applies the course to a dataset it did not teach on.
- Lab · 180 minThe interval the survey actually boughtA 30-cluster SMART survey with no weights and no frame. Work out what it can and cannot support, compute GAM with a design-adjusted interval, and decide whether it can be read against the 15% emergency threshold.
- Exercise · 60 minThe weight that was never appliedA colleague's analysis script produces four estimates from the household survey. Every one is computed correctly and every one is wrong, in four different ways. Find each, fix it, and quantify what it cost.
Progress
Take it offline
The whole course as a typeset PDF — every lesson, every code example, the data dictionary and the indicator definitions. Generated from the same source as this page.
The LaTeX source ships alongside each PDF, so an organisation can rebrand the handout or fold a lesson into its own training pack.