Four decisions, one coefficient

A rural water programme wants to know which management model keeps water points working. The register has 2,629 visits with a functional status on each, a management model on each point, and every covariate a model would want.

Fitting that model is four lines. Getting it right is four decisions, and this lab is those decisions made explicitly and in order — which covariates, which unit, which weights, and what to put in the report.

The file

water-point-monitoring-2024.v1.csv — 2,629 monitoring visits to 242 water points across 18 communities and 3 districts, twelve monthly rounds. Synthetic.

Set up first

A virtual environment, the file read-only, an outputs/ directory, and a script that runs top to bottom. Every number in the report you hand in is printed by that script.

Decision one: what the outcome is

functional_status has four values, not two. Before any model exists, decide what “working” means and write the decision down.

print(points["functional_status"].value_counts())

1,823 functional, 512 non-functional, 160 abandoned, 134 partially functional.

Three defensible outcomes and they are different questions:

• functional against everything else — is water available today;
• functional or partially-functional against the rest — is the point in service;
• dropping abandoned entirely — of the points still in the programme, how many work.

Pick one, state why in a comment, and use it throughout. The WASH course established that a water point programme has three functionality rates rather than one; this is the same fact arriving as a modelling decision.

Decision two: which covariates

Build the model for the management question, not for fit.

Available: management, source_type, installed_year, users_estimated, district, community, round, fee_collected, queue_minutes, days_since_breakdown.

Three of those must not be in a total-effect model for management, and you have to say why. Work out which before reading on — the test from lesson 5 is whether the variable is caused by the exposure, caused by the outcome, or both.

model = smf.logit(
    "working ~ management + source_type + age + district", data=points).fit()

Then justify each inclusion in one line: confounder, competing cause, or design variable. A covariate you cannot classify does not go in.

Decision three: what the unit is

This is the decision the register makes unavoidable, and it is not the one from the lessons.

print(f"{len(points)} visits, {points['water_point_id'].nunique()} points")
print(points.groupby("water_point_id").size().describe())

242 points, visited a median of eleven times. The rows are not 2,625 independent observations of a management model — they are 242 water points observed repeatedly, and a point that is broken in March is likely to be broken in April.

Fit it three ways and put them side by side:

• the naive visit-level logistic model;
• the same model with standard errors clustered on water_point_id;
• a point-level linear model on each point’s proportion of rounds working.

Every coefficient in the first two is identical. Only the standard errors move, and they move by different amounts for different terms — record the ratio for each.

Decision four: what the report says

Produce one table with the coefficient, both standard errors, and a column saying whether the conclusion changes. Then write the four-sentence claim from the statistics course for the management comparison you consider best supported.

Check your numbers

	Expected
Visits, points, communities	2,629 · 242 · 18
Working (functional only), visit level	about 69.3%
Working, point level	about 69.2%
Private operator, naive OR	about 1.68, 95% CI 1.29 to 2.19
Private operator, clustered OR	1.68, 95% CI 1.10 to 2.56
NGO-managed, naive OR	about 1.44, z = 2.59
NGO-managed, clustered OR	1.44, z = 1.61
Point age, naive	OR 0.976 per year, z = −2.99
Point age, clustered	OR 0.976 per year, z = −1.65
Standard error inflation	between 1.1× and 2.0× depending on the term

Four terms cross the 0.05 line when the clustering is corrected — NGO management, protected springs, point age and the Sud-Est district — and not one coefficient changes. If your coefficients moved, you refitted rather than re-estimating the covariance.

If your point-level model disagrees with the clustered one about which terms survive, check whether you aggregated the covariates with first — a point’s management model does not change between rounds, and taking a mean of a string silently drops the column.

The questions to answer in prose

Three sentences each.

1. Name the three covariates you excluded and classify each one — mediator, collider, or downstream of the outcome. For the one you found hardest, say what would have to be true about the programme for your classification to be wrong.

2. Four terms lose significance under clustering and none of the coefficients move. Explain to a WASH coordinator, without using the word “variance”, why the estimate is unchanged and the conclusion is not.

3. The naive model has 2,625 rows and the point-level model has 242. State the effective sample size the clustered model implies, and say what the register would have to look like for 2,625 to be the honest number.

What to hand in

A Python script, run from a clean environment, producing:

• the outcome definition as a comment at the head of the file, with its reason
• the covariate table: each variable, in or out, and its classification
• the three model fits in one table, with both standard errors and the ratio
• the point-level fit as the third column
• a report.txt in the shape of the “report it whole” blocks, with the management comparison written as four sentences and a limitations paragraph naming the clustering, the observational design and the outcome definition
• the three prose answers as a comment block at the foot

How to know you are done

Delete outputs/, rerun, everything regenerates identically. Then change the outcome definition at the top of the script from functional to functional or partially-functional: every number in the report should move and the structure of the report — which terms survive clustering — should not. If changing one constant at the top does not propagate everywhere, a number was typed rather than computed.

What this lab is not

It is not a survival analysis of breakdowns, and it is not a model of days_since_breakdown — that variable is downstream of the outcome and belongs to a different question. It is also not an evaluation of management models: nobody randomised water points into private operation, and the private operator coefficient is a comparison of points that already differed. What this lab is about is that the four decisions are made whether or not you make them — a default fit makes all four badly and prints a table that looks exactly like a good one.