---
title: "Pick the right denominator — worked solution"
subtitle: "Exercise solution · Data Analysis Foundations for M&E"
author: "Cassion · data-analysis.cassion.dev"
format:
  html:
    toc: true
    code-fold: false
jupyter: python3
---

## How to use this

This is the solution, so read it after attempting the exercise rather than
instead of. The five scenarios each have an agreed numerator and a contested
denominator, and in every one the honest answer is not "the right number" but
"the number, plus what it excludes".

The dataset is the synthetic MUAC screening register from Artibonite. Nothing
here describes a real child.

```{python}
import pandas as pd
import numpy as np

URL = (
    "https://data-analysis.cassion.dev/datasets/files/"
    "muac-screening-artibonite-2024.v1.csv"
)

muac = pd.read_csv(
    URL,
    dtype={"child_id": "string", "commune": "string", "sex": "string"},
    na_values={"muac_mm": ["-99"]},
)
muac["screening_date"] = pd.to_datetime(muac["screening_date"], format="%Y-%m-%d")

# The corrections from lesson 6, applied so the scenarios below argue about
# denominators rather than about data quality.
unit_error = muac["muac_mm"].notna() & (muac["muac_mm"] < 40)
muac.loc[unit_error, "muac_mm"] = muac.loc[unit_error, "muac_mm"] * 10

oedema_map = {
    "true": True, "TRUE": True, "Y": True, "y": True, "yes": True,
    "false": False, "FALSE": False, "N": False, "n": False, "no": False,
}
muac["oedema"] = muac["oedema"].astype("string").str.strip().map(oedema_map)
muac = muac.drop_duplicates()

muac["gam"] = (muac["muac_mm"] < 125) | (muac["oedema"] == True)
len(muac)
```

## Scenario 1 — "What proportion of children screened were malnourished?"

The numerator is agreed: children meeting the GAM case definition. Four
denominators are defensible and they give four different numbers.

```{python}
candidates = {
    "all rows in the register": len(muac),
    "rows with a MUAC measurement": int(muac["muac_mm"].notna().sum()),
    "rows with MUAC or an oedema assessment": int(
        (muac["muac_mm"].notna() | muac["oedema"].notna()).sum()
    ),
    "rows with MUAC, oedema and age": int(
        (
            (muac["muac_mm"].notna() | muac["oedema"].notna())
            & muac["age_months"].notna()
        ).sum()
    ),
}

cases = int(muac["gam"].sum())
pd.DataFrame(
    {
        "denominator": candidates,
        "rate": {k: cases / v for k, v in candidates.items()},
    }
).round(4)
```

**The answer.** Use "rows with MUAC or an oedema assessment". A child with no
assessment at all was not screened for this indicator, so including them in the
denominator understates the rate by counting non-observations as negatives.
Requiring age as well is over-strict: the MUAC thresholds for 6 to 59 months are
a single band and do not use age, and demanding it drops one commune far more
than the others.

**What it excludes, stated out loud:** children who did not come to be screened.
This is a rate among those reached, not a population prevalence, and the report
has to say so.

## Scenario 2 — "What is our referral completion rate?"

The trap is that the decision column and the measurement column were filled by
different people, so a handful of records carry a referral with no measurement
behind it. Count them rather than assuming the number.

```{python}
referred = muac["outcome"].isin(["referred-tsfp", "referred-otp", "referred-sc"])
no_measurement = referred & muac["muac_mm"].isna()

print(f"referrals recorded          : {int(referred.sum())}")
print(f"of which with no measurement: {int(no_measurement.sum())}")
```

**The answer.** There are two different questions hiding here, and they need
different denominators.

- *Did the screening lead to a referral where it should have?* Denominator:
  children meeting the referral case definition. This measures the screener.
- *Did the referred child reach the service?* Denominator: children referred.
  This measures the pathway, and this register cannot answer it at all — there
  is no arrival record.

Reporting the second using this file would be inventing a number. The correct
output is the first, plus a sentence saying the second requires the CMAM
admission register.

## Scenario 3 — "Coverage went up 12% this quarter"

```{python}
muac["quarter"] = muac["screening_date"].dt.to_period("Q")
by_quarter = muac.groupby("quarter").size()
by_quarter
```

**The answer.** This is not coverage, and the word should be refused. Coverage is
cases reached over cases existing, and this register has no denominator of
children in the population — only those who turned up. What went up is screening
volume.

Note also that Q1 and Q4 are short: the first screening is mid-January and the
last mid-December, so a quarter-on-quarter comparison including either is
comparing unequal windows.

```{python}
print("first:", muac["screening_date"].min().date())
print("last :", muac["screening_date"].max().date())
```

## Scenario 4 — "Which commune has the worst malnutrition?"

```{python}
by_commune = (
    muac.assign(assessed=muac["muac_mm"].notna() | muac["oedema"].notna())
    .groupby("commune")
    .agg(assessed=("assessed", "sum"), cases=("gam", "sum"))
)
by_commune["rate"] = by_commune["cases"] / by_commune["assessed"]
by_commune.sort_values("rate", ascending=False).round(4)
```

**The answer.** The denominator is right, and the question is still wrong. The
smallest commune has under 200 assessments, so its interval is wide enough that
its rank is close to meaningless.

```{python}
from scipy.stats import beta

def interval(cases, n):
    if n == 0:
        return (np.nan, np.nan)
    lower = beta.ppf(0.025, cases, n - cases + 1) if cases > 0 else 0.0
    upper = beta.ppf(0.975, cases + 1, n - cases) if cases < n else 1.0
    return (lower, upper)

bounds = by_commune.apply(lambda r: interval(r["cases"], r["assessed"]), axis=1)
by_commune["low"] = [b[0] for b in bounds]
by_commune["high"] = [b[1] for b in bounds]
by_commune.sort_values("rate", ascending=False)[
    ["assessed", "rate", "low", "high"]
].round(4)
```

Several intervals overlap. "Worst" is answerable only for the commune whose
interval clears the others, and a table without intervals would have let you
rank all twelve with false confidence.

## Scenario 5 — "Our programme reached 4,218 children this year"

```{python}
print(f"rows                    : {len(muac)}")
print(f"distinct child_id       : {muac['child_id'].nunique()}")

key = ["commune", "screening_date", "age_months", "sex", "muac_mm"]
suspected = muac.duplicated(subset=key, keep=False) & muac["muac_mm"].notna()
print(f"suspected re-registration: {int(suspected.sum())}")
```

**The answer.** Rows are not children. Exact duplicates have already been
removed above, but a handful of children were re-registered under a new
identifier and share no key — findable only by matching on commune, date, age,
sex and measurement, and indistinguishable from genuine coincidence without the
paper register.

So the defensible statement is a figure with its caveat attached: so many
distinct registrations, of which some number may be re-registrations of the same
child. Note that the check above flags *suspects*, not duplicates — two children
of the same age and sex screened in one commune on one day with the same
measurement is entirely possible, and in a campaign this size it is likely.
Resolving them needs the paper register, not another line of pandas.

## The rule underneath all five

Write the denominator as a sentence before you compute anything. If you cannot,
you do not yet have an indicator — you have a column you are about to average.
And whatever the denominator excludes goes in the report, not in your head.