The district with no open defecation

A sanitation table is circulating. It has six districts in it, one of which reports no open defecation at all, and a pairwise test says that district differs significantly from the worst performer at p = 0.027. A hygiene programme is being asked why one district solved the problem and the others did not.

Everything in that paragraph is arithmetically correct and none of it is true. This lab is the whole course applied to one table: normalise before you compare, put an interval on a zero, count the comparisons you ran, and check the unit of analysis before you believe any of it.

The file

wash-household-survey-2024.v1.csv — 2,403 households across three districts and eighteen communities, with sanitation, water treatment and water quality. Synthetic.

Set up first

An RStudio project, the file read-only, an outputs/ directory, and a script that runs top to bottom from a clean session. Every number in this lab is printed by that script; none is typed.

Part one: the table as it arrived

Build the open defecation table exactly as the circulating one was built — group by the district column as it is stored, and add a Wilson interval to every row.

library(dplyr)

wash |>
  summarise(
    k = sum(sanitation_facility == "open-defecation"),
    n = n(),
    .by = district
  ) |>
  rowwise() |>
  mutate(
    rate = k / n,
    lower = prop.test(k, n)$conf.int[1],
    upper = prop.test(k, n)$conf.int[2]
  )

You will get six rows, four of which are Nord-Ouest written four different ways, with sample sizes of 727, 33, 22 and 20.

Before going on, write down what the 20-household row’s interval is — the rate is 0% and the Wilson interval runs to about 16%. Compute the Wald interval on the same cell and note that it is [0, 0]. One of those two numbers would let someone claim a solved district; the other would not.

Part two: the comparisons the table invites

Run every pairwise comparison between the six rows — there are fifteen — and record the p-value of each.

# combn() over the six rows, prop.test() on each pair.

One pair comes back significant at p = 0.027: the 33-household NORD-OUEST against the 20-household nord-ouest, a gap of 21.2 points between two spellings of the same place.

Apply the correction from lesson 5. Fifteen tests, Bonferroni threshold 0.05/15 = 0.0033, and nothing survives. Record the expected number of false positives at the uncorrected threshold and compare it with what you found.

Part three: normalise, and run it again

Collapse the district codes — trim, lowercase, replace the space — and rebuild both the table and the family of comparisons.

wash <- wash |> mutate(district = gsub(" ", "-", tolower(trimws(district))))

District	Open defecation	n
centre	?	798
nord-ouest	?	802
sud-est	?	803

Three rows, three comparisons, and the largest difference between any two districts is 0.21 percentage points. Every p-value is above 0.90.

Write one sentence contrasting the two versions of this analysis. The first produced a significant finding about a district that does not exist; the second produces the correct answer, which is that open defecation is uniform across the three districts at about 13.3%.

Part four: a difference that is actually there

Now do the same for water treatment at home, which is a different story entirely.

wash |>
  summarise(k = sum(water_treated_at_home), n = n(), .by = district)

Three districts, three pairwise comparisons, and all three survive the Bonferroni correction with room to spare. Report each with its interval and its difference, and note that the smallest of the three gaps is 12.2 points — an effect size that a hygiene promotion budget can be allocated against.

This is the contrast the lab exists for. The same procedure that dissolved the sanitation finding leaves this one standing, which is what makes the dissolution credible rather than defensive.

Part five: eighteen communities, not 2,403 households

The survey is a cluster sample: eighteen communities, roughly 133 households each. Apply lesson 6.

by_community <- wash |>
  summarise(rate = mean(water_treated_at_home), .by = c(district, community))

Compute the intra-cluster correlation for water treatment, the design effect, and the effective sample size. Then produce two intervals on the overall treatment rate: the household-level one, and one built from the eighteen community rates.

The community-level standard error is about 2.8 times the household-level one, and the interval on the overall rate widens from roughly ±2 points to roughly ±6.

Re-run the three district comparisons at community level — six communities per district, Welch’s t-test — and confirm that all three differences survive there too. A finding that survives the honest standard error is one you can defend; the point of computing it is not to weaken the result but to know whether it needs to be.

Check your numbers

	Expected
Districts in the raw column	6
Districts after normalising	3
Open defecation, whole sample	319 of 2,403, about 13.3%
Largest district gap, open defecation	about 0.2 points, p > 0.9
Water treatment, centre / sud-est / nord-ouest	about 56.4% / 44.2% / 29.4%
Smallest water-treatment gap	12.2 points, p < 0.001
Intra-cluster correlation, water treatment	about 0.05
Design effect, and effective n	about 7.9, so about 304

If your open-defecation rate is far from 13.3%, the sanitation categories are being matched against a value that is not in the column. If your ICC comes out negative, you have computed it on open defecation rather than water treatment — which is a real result and belongs in your write-up.

The questions to answer in prose

Three sentences each.

1. The circulating table produced a significant difference at p = 0.027. Name every reason that result should not have reached a report, in the order a reviewer would find them.

2. The 20-household row shows 0% open defecation. Explain to a programme manager why “no households practise open defecation here” is not what that row says, using the interval rather than the word “sample size”.

3. Water treatment differs across districts by up to 27 points and open defecation does not differ at all. Both were computed from the same 2,403 rows. Describe what a district-targeted programme should conclude from that pair of results.

What to hand in

An R script, sourced from a clean session, producing:

• both district tables, raw and normalised, each row with its Wilson interval
• the fifteen-comparison family and the three-comparison family, each with its test count and its corrected threshold stated
• the water-treatment table with pairwise comparisons at household and community level
• the ICC, design effect and effective sample size
• a report.txt written in the shape of the “report it whole” blocks: findings in the body, tests in the annex, the total number of comparisons run on the last line
• the three prose answers as a comment block at the foot

How to know you are done

Delete outputs/, restart R, one source(), everything regenerates identically. Then change the normalisation to a simple tolower() without the space replacement: Nord Ouest splits back out, you are testing four districts again, and the number of comparisons in your report changes on its own. If it does not, the test count is hard-coded, and lesson 5 was about the one number a report must never hard-code.

What this lab is not

It is not a data cleaning lab — the WASH course already taught the inconsistent coding defect, and here it is simply the mechanism. It is not a survey design lab either: the design effect is measured from the realised sample rather than planned for, and planning it is what Survey Analysis and Sampling is for. What this lab is about is the order of operations. Normalise, then estimate, then compare, then correct, then check the unit — and every step skipped in that sequence produced a finding in part one that part three had to take away.