Missingness that is not an accident

Slide 1 / 22

What this lesson covers

• 5.4% is not a finding
• Break it down by group before you do anything else
• Then by time, inside the group that stands out
• Name the mechanism, in words the report can carry
• Put a number on what dropping them costs
• The three responses, and when each is honest
• Missing rows, not just missing values
• What comes next

Slide 2 / 22

5.4% is not a finding

• A completeness figure only means something once you know where the holes are — Scattered missingness costs precision

Slide 3 / 22

Break it down by group before you do anything else — In Python

by_commune = (
    muac.assign(missing_age=muac["age_months"].isna())
    .groupby("commune")
    .agg(missing=("missing_age", "mean"), n=("missing_age", "size"))
    .sort_values("missing", ascending=False)
)
print((by_commune["missing"] * 100).round(1))

Slide 4 / 22

Break it down by group before you do anything else — In R

muac |>
  group_by(commune) |>
  summarise(missing = mean(is.na(age_months)), n = n()) |>
  arrange(desc(missing)) |>
  mutate(missing = round(100 * missing, 1))

Slide 5 / 22

Break it down by group before you do anything else

Slide 6 / 22

Then by time, inside the group that stands out — In Python

gros_morne = muac[muac["commune"] == "Gros-Morne"].copy()
gros_morne["week"] = gros_morne["screening_date"].dt.to_period("W").dt.start_time

print(
    gros_morne.groupby("week")["age_months"]
    .apply(lambda s: s.isna().mean())
    .sort_values(ascending=False)
    .head()
)

Slide 7 / 22

Then by time, inside the group that stands out — In R

muac |>
  filter(commune == "Gros-Morne") |>
  mutate(week = lubridate::floor_date(screening_date, "week", week_start = 1)) |>
  group_by(week) |>
  summarise(missing = mean(is.na(age_months)), n = n()) |>
  arrange(desc(missing)) |>
  head()

Slide 8 / 22

Name the mechanism, in words the report can carry

• Missing completely at random. The holes are unrelated to anything, including the value that is missing. Dropping…
• Missing at random. The holes depend on something you observed — a commune, a week, an enumerator. Dropping them…
• Missing not at random. The holes depend on the missing value itself. The sickest children are the ones the queue…

Slide 9 / 22

Name the mechanism, in words the report can carry

The category is not a statistical technicality. It is the difference between "we dropped 226 rows" and "we dropped 226 rows, 54 of them from one commune in one week, which moved that commune's rate by 1.1 points".

Slide 10 / 22

Put a number on what dropping them costs — In Python

def gam_rate(df):
    assessed = df["muac_mm"].notna() | df["oedema"].notna()
    cases = assessed & ((df["muac_mm"] < 125) | (df["oedema"] == True))
    return pd.Series({
        "assessed": int(assessed.sum()),
        "cases": int(cases.sum()),
        "rate": round(cases.sum() / assessed.sum(), 3),
    })


all_rows = muac.groupby("commune").apply(gam_rate)
complete = muac[muac["age_months"].notna()].groupby("commune").apply(gam_rate)

comparison = all_rows.join(complete, rsuffix="_complete")
comparison["shift"] = comparison["rate_complete"] - comparison["rate"]
print(comparison.sort_values("shift"))

Slide 11 / 22

Put a number on what dropping them costs — In R (cont.)

gam_rate <- function(df) {
  df |>
    mutate(
      assessed = !is.na(muac_mm) | !is.na(oedema),
      case     = assessed & (muac_mm < 125 | oedema)
    ) |>
    summarise(
      assessed = sum(assessed, na.rm = TRUE),
      cases    = sum(case, na.rm = TRUE),
      rate     = round(cases / assessed, 3),
      .by      = commune
    )
}

comparison <- gam_rate(muac) |>
  left_join(gam_rate(filter(muac, !is.na(age_months))),

Slide 12 / 22

Put a number on what dropping them costs — In R (cont.)

            by = "commune", suffix = c("", "_complete")) |>
  mutate(shift = rate_complete - rate) |>
  arrange(shift)

Slide 13 / 22

Put a number on what dropping them costs

Slide 14 / 22

The three responses, and when each is honest

• Drop them, and report the drop — Legitimate when the missingness is scattered and you have shown it is
• Keep them, in a category of their own — Often the right answer for a categorical variable, because "not recorded" is a…
• Impute — carefully, and rarely — For a variable used to disaggregate rather than to compute, filling missing age from…

Slide 15 / 22

The three responses, and when each is honest — In Python

muac["age_imputed"] = muac["age_months"].isna()
muac["age_months_filled"] = muac.groupby("commune")["age_months"].transform(
    lambda s: s.fillna(s.median())
)

Slide 16 / 22

The three responses, and when each is honest — In R

muac <- muac |>
  mutate(age_imputed = is.na(age_months)) |>
  group_by(commune) |>
  mutate(age_months_filled = ifelse(is.na(age_months),
                                    median(age_months, na.rm = TRUE),
                                    age_months)) |>
  ungroup()

Slide 17 / 22

The three responses, and when each is honest

• The flag column is the point — An imputed value that is indistinguishable from a measured one has stopped being an…

Slide 18 / 22

Missing rows, not just missing values

Slide 19 / 22

Missing rows, not just missing values — In Python

vax["reported"] = vax["report_submitted"] == True

reporting_rate = vax.groupby("period")["reported"].mean()
coverage = (
    vax[vax["reported"]]
    .groupby("period")
    .apply(lambda g: g["doses_administered"].sum() / g["target_population"].sum())
)
print(pd.DataFrame({"reporting_rate": reporting_rate, "coverage": coverage}))

Slide 20 / 22

Missing rows, not just missing values — In R

vax |>
  summarise(
    reporting_rate = mean(report_submitted),
    coverage = sum(doses_administered[report_submitted]) /
               sum(target_population[report_submitted]),
    .by = period
  )

Slide 21 / 22

What comes next

• Missingness is a hole where something should be.

Slide 22 / 22

Where this goes next

Read the full lesson, with runnable code Back to the lesson