R² of 0.03, and the most useful result in the report

A model built to answer a targeting question

A programme wants to screen fewer children. If household characteristics predicted which children are malnourished, screening could be targeted at the households most likely to hold them, and the survey carries every variable such a rule would use.

import pandas as pd
import numpy as np
import statsmodels.formula.api as smf

survey = pd.read_csv("household-survey-2025.v1.csv")
d = survey.dropna(subset=["child_muac_mm", "child_age_months", "child_sex",
                          "household_size", "displacement_status",
                          "main_livelihood", "food_insecure",
                          "improved_water_source"])

model = smf.ols(
    "child_muac_mm ~ child_age_months + child_sex + household_size"
    " + displacement_status + main_livelihood + food_insecure"
    " + improved_water_source", data=d).fit()

print(f"n = {int(model.nobs)}   R2 = {model.rsquared:.4f}"
      f"   adjusted R2 = {model.rsquared_adj:.4f}")
print(f"residual SD {np.sqrt(model.scale):.2f}  outcome SD {d['child_muac_mm'].std():.2f}")

model <- lm(child_muac_mm ~ child_age_months + child_sex + household_size +
              displacement_status + main_livelihood + food_insecure +
              improved_water_source, data = d)
summary(model)

n = 641. R² = 0.030, adjusted R² = 0.011. Residual SD 15.95 mm against an outcome SD of 16.04 mm.

The model has removed less than one millimetre of the sixteen it started with. One coefficient is distinguishable from zero.

Term	Coefficient	95% CI
Food insecure	−4.70 mm	−7.45 to −1.95
Salaried livelihood	−3.49 mm	−8.44 to +1.45
Farming livelihood	−2.76 mm	−6.16 to +0.64
Child age, per month	−0.03 mm	−0.11 to +0.05
Improved water source	−1.09 mm	−3.76 to +1.58

Do not delete it

The instinct is to drop the model, try more variables, or reach for something non-linear. All three are wrong here, and the reason is that the question was not “can I model MUAC” — it was “can I target screening”.

d = d.assign(predicted=model.fittedvalues, mam=(d["child_muac_mm"] < 125))
for frac in (0.20, 0.30, 0.50):
    cut = d["predicted"].quantile(frac)
    caught = d.loc[d["predicted"] <= cut, "mam"].sum()
    print(f"screen worst {frac:.0%}: catches {caught}/{d['mam'].sum()}"
          f" = {caught / d['mam'].sum():.1%} of cases")

d |> mutate(predicted = fitted(model), mam = child_muac_mm < 125) |>
  arrange(predicted) |>
  summarise(caught = mean(head(mam, n() * 0.2)) * sum(mam))

Screening rule	Children screened	MUAC < 125 mm found
Worst-predicted 20%	129	28 of 90 = 31.1%
Worst-predicted 30%	193	41 of 90 = 45.6%
Worst-predicted 50%	321	54 of 90 = 60.0%
Random 20%	129	20% expected

Targeting on the model finds 31% of the malnourished children while screening 20% of them. Screening 20% at random finds 20%. The model is eleven points better than chance and misses seven in ten of the children a programme exists to reach.

That is the finding. Not “the model did not work” — household characteristics do not identify malnourished children well enough to target screening, so screening has to be blanket. A programme can act on that sentence, it costs a real amount of money, and it is supported by a model with an R² of 0.03.

Diagnostics that change a decision

Four checks, and each one has a decision attached rather than a threshold.

Fitted values outside the possible range. For a binary outcome fitted with linear regression, this is immediate.

lpm = smf.ols("completed ~ disability + case_category + age_band + sex"
              " + service_requested + admin1", data=referrals).fit()
print(f"fitted values from {lpm.fittedvalues.min():.3f}"
      f" to {lpm.fittedvalues.max():.3f}")
print(f"outside [0, 1]: {((lpm.fittedvalues < 0) | (lpm.fittedvalues > 1)).sum()}")

range(fitted(lm(completed ~ ., data = referrals)))

Eight of 1,581 fitted probabilities fall below zero, the lowest at −0.060. The logistic model on the same data stays within 0.065 and 0.738. The decision: use logistic when predictions matter, and the linear probability model when only the average difference does — the linear model’s coefficient is directly a risk difference, which is why it survives in economics.

Residuals against fitted values. Look for a fan shape, which means the spread depends on the level.

import matplotlib.pyplot as plt
plt.scatter(model.fittedvalues, model.resid, s=6)

plot(model, which = 1)

The decision: heteroscedasticity does not bias the coefficient, it biases the standard error — so the fix is a robust standard error, not a transformed outcome.

Influence. A handful of rows can carry a coefficient.

influence = model.get_influence().cooks_distance[0]
print(f"rows with Cook's D > 4/n: {(influence > 4 / len(d)).sum()}")

sum(cooks.distance(model) > 4 / nobs(model))

The decision: look at the flagged rows before doing anything. The WASH course’s eleven households with a unit error are exactly the kind of row that turns up here, and the answer is to fix the data, not to down-weight the point.

Linearity of a continuous predictor. Fit it in bands and see whether the coefficients step evenly.

d["age_band"] = pd.cut(d["child_age_months"], [0, 12, 24, 36, 48, 60])
print(smf.ols("child_muac_mm ~ C(age_band)", data=d).fit().params.round(2))

lm(child_muac_mm ~ cut(child_age_months, c(0, 12, 24, 36, 48, 60)), data = d)

The decision: if the bands do not step evenly, the linear term is answering the wrong question, and banding is usually a better answer than a polynomial because the bands are interpretable to the people reading the report.

What R² is and is not

R² is the share of variance the model accounts for. In programme data it is routinely small and that is a fact about people, not about the analyst.

It is not a measure of whether a coefficient is right. A randomised trial with a decisive treatment effect can have an R² of 0.02, because individual variation swamps a real average difference. Judging a causal estimate by R² is a category error.

It is a measure of whether predictions are worth making. That is the use above, and it is the honest one: the targeting question is a prediction question, and R² answered it.

Adjusted R² penalises added terms, and it went from 0.030 to 0.011 here — the gap between the two is a measure of how many of the twelve variables were paying for themselves. Almost none were.

Report it whole

Predicting child MUAC from household characteristics

  Linear model, 641 children with complete data.
  R-squared 0.030, adjusted 0.011. Residual SD 15.95 mm against an outcome
  SD of 16.04 mm.

  Only food insecurity is distinguishable from zero: -4.70 mm (95% CI -7.45
  to -1.95). Household size, displacement status, water source, child sex
  and child age are not.

  Used as a targeting rule, screening the 20% of children the model ranks
  worst would find 31% of the children with MUAC below 125 mm, against 20%
  expected from screening 20% at random. Screening half the children would
  find 60%.

  Recommendation: do not target screening on household characteristics. The
  survey's 14.0% prevalence of MUAC below 125 mm is not concentrated in
  households that a registration form can identify, and blanket screening
  remains the only rule that reaches the caseload.

  This model is reported because it does not fit. A negative result about
  targeting is a budget decision, and dropping the model would have left the
  proposal to assume targeting works.

The last paragraph is why the section exists. A model that does not fit is evidence, and the file drawer it usually goes into is where a programme’s assumption survives unchallenged.

What comes next

That is the course. Eight lessons, one idea: a coefficient is a comparison, and the work is saying which comparison, between which units, adjusted for what, with a standard error that matches how the data arrived.

The lab puts all four decisions in one analysis. Then Impact Evaluation Methods takes on the question this course has deferred in every lesson’s last paragraph — which of these comparisons can be written as a cause.