A script another officer can run — Python for Programme Data — Data Analysis

The handover test

An analysis is finished when someone else can run it on next quarter’s export without editing the code. Not “without much editing” — without editing.

That standard rules out the three habits every notebook accumulates: a path written for one machine, a value changed by hand between runs, and a step that only works if you already know which cell to run first.

Arguments, not edits

# scripts/indicator_table.py
import argparse
from pathlib import Path

def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Compute GAM and SAM rates by commune from a screening register."
    )
    parser.add_argument(
        "--input", type=Path, required=True,
        help="Path to the screening register CSV.",
    )
    parser.add_argument(
        "--output", type=Path, required=True,
        help="Directory to write tables into. Created if missing.",
    )
    parser.add_argument(
        "--gam-threshold", type=int, default=125,
        help="MUAC cut-off in mm for global acute malnutrition (default: 125).",
    )
    parser.add_argument(
        "--min-assessment-rate", type=float, default=0.8,
        help="Communes below this assessment rate are flagged, not dropped.",
    )
    return parser.parse_args()

uv run python scripts/indicator_table.py \
    --input data/raw/muac-screening-artibonite-2024.v1.csv \
    --output outputs/tables

type=Path converts the string for you, and required=True means a forgotten argument produces a clear message rather than a KeyError two hundred lines in.

Put the thresholds in arguments and the defaults in the code. A threshold that is only ever changed by editing a line will eventually be changed and not changed back. One that is an argument with a documented default is visible in the command that produced the output, which is what you want when someone asks how last quarter’s figure was computed.

--help is generated from the same declarations, and it is the documentation people actually read:

uv run python scripts/indicator_table.py --help

Fail loudly, early

A script that produces a wrong number silently is worse than one that crashes. Check what you assume, at the point you assume it.

def read_register(path: Path) -> pd.DataFrame:
    if not path.exists():
        raise SystemExit(f"Input file not found: {path}")

    muac = pd.read_csv(
        path,
        dtype={"child_id": "string", "commune": "string"},
        na_values={"muac_mm": ["-99"]},
    )

    expected = {"child_id", "commune", "screening_date", "muac_mm", "oedema"}
    missing = expected - set(muac.columns)
    if missing:
        raise SystemExit(f"Input is missing columns: {sorted(missing)}")

    if muac["child_id"].isna().any():
        raise SystemExit("Some rows have no child_id; refusing to continue.")

    return muac

SystemExit with a message exits with a non-zero status and prints one readable line, rather than a traceback the reader has to interpret. Use it for “this input is not what I was promised”; use assert for “this cannot happen unless the code is wrong”.

Turn the pandas warning that means a silent failure into an error:

import warnings
import pandas as pd

warnings.simplefilter("error", pd.errors.ChainedAssignmentError)

Chained assignment never modifies the frame, and by default it only warns. In a script whose output someone will report, stopping is the right response.

Logging, not print

print goes to standard output alongside the results, has no severity, and cannot be turned down.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)-8s %(message)s",
    datefmt="%H:%M:%S",
)
log = logging.getLogger(__name__)

log.info("Read %d rows from %s", len(muac), args.input)
log.warning("%d communes below the assessment-rate floor", len(flagged))
log.info("Wrote %s", output_path)

09:14:02 INFO     Read 4218 rows from data/raw/muac-...v1.csv
09:14:02 WARNING  2 communes below the assessment-rate floor
09:14:02 INFO     Wrote outputs/tables/gam_by_commune.csv

Three things this buys that print does not: a timestamp, a severity someone can filter on, and the ability to add --verbose without touching every call.

Use the %s form rather than an f-string — the formatting is skipped entirely when the level is disabled.

Log the numbers that would let someone reconstruct the run: rows read, rows excluded and why, the denominator, the output path. A log that says “done” tells nobody anything.

The shape of the script

# scripts/indicator_table.py
"""Compute GAM and SAM rates by commune from a MUAC screening register."""

import argparse
import logging
from pathlib import Path

import pandas as pd

log = logging.getLogger(__name__)

GAM_DEFAULT, SAM_DEFAULT = 125, 115


def read_register(path: Path) -> pd.DataFrame:
    ...


def indicator_table(muac: pd.DataFrame, gam_mm: int, sam_mm: int) -> pd.DataFrame:
    measured = muac["muac_mm"].notna() | muac["oedema"].notna()
    gam = measured & ((muac["muac_mm"] < gam_mm) | (muac["oedema"] == True))
    sam = measured & ((muac["muac_mm"] < sam_mm) | (muac["oedema"] == True))

    table = (
        muac.assign(measured=measured, gam=gam, sam=sam)
        .groupby("commune", dropna=False, observed=True)
        .agg(
            screened=("child_id", "size"),
            assessed=("measured", "sum"),
            gam_cases=("gam", "sum"),
            sam_cases=("sam", "sum"),
        )
    )
    table["assessment_rate"] = (table["assessed"] / table["screened"]).round(3)
    table["gam_rate"] = (table["gam_cases"] / table["assessed"]).round(3)
    table["sam_rate"] = (table["sam_cases"] / table["assessed"]).round(3)
    return table.reset_index()


def main() -> None:
    args = parse_args()
    logging.basicConfig(level=logging.INFO, format="%(levelname)-8s %(message)s")

    muac = read_register(args.input)
    log.info("Read %d rows", len(muac))

    table = indicator_table(muac, args.gam_threshold, SAM_DEFAULT)

    thin = table.loc[table["assessment_rate"] < args.min_assessment_rate, "commune"]
    if len(thin):
        log.warning("Assessment rate below floor: %s", ", ".join(thin))

    args.output.mkdir(parents=True, exist_ok=True)
    destination = args.output / "gam_by_commune.csv"
    table.to_csv(destination, index=False)
    log.info("Wrote %s (%d communes)", destination, len(table))


if __name__ == "__main__":
    main()

Four things about that structure are deliberate.

Functions take arguments and return values. indicator_table does not read a file, does not know where output goes, and does not touch a global. That is what makes it testable and what lets the notebook import it.

main() is the only place that does input and output. Reading, writing and logging live there; everything else is computation.

The if __name__ == "__main__" guard means importing this module from a notebook runs nothing. Without it, from indicator_table import indicator_table executes the whole script.

Thin data is flagged, not dropped. A commune below the assessment-rate floor stays in the table with a warning beside it. Dropping it would change the district denominator and nothing in the output would say so.

Testing the part that computes

Once the computation is a function taking a frame, testing it is three lines:

# tests/test_indicator_table.py
import pandas as pd
from indicator_table import indicator_table


def test_denominator_excludes_unmeasured_children():
    muac = pd.DataFrame({
        "child_id": ["a", "b", "c"],
        "commune": ["X", "X", "X"],
        "muac_mm": [110, 130, None],
        "oedema": [False, False, None],
    })

    table = indicator_table(muac, gam_mm=125, sam_mm=115)

    assert table.loc[0, "screened"] == 3
    assert table.loc[0, "assessed"] == 2      # the unmeasured child is not a denominator
    assert table.loc[0, "gam_rate"] == 0.5

That test encodes the decision from lesson 6 — which children are in the denominator — in a form that fails if someone changes it. It is worth writing for every indicator whose definition you had to argue about.

Making the run reproducible

Record what produced the output, beside the output:

import json
import subprocess
from datetime import datetime, timezone


def run_metadata(args) -> dict:
    return {
        "generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
        "input": str(args.input),
        "gam_threshold": args.gam_threshold,
        "pandas": pd.__version__,
        "commit": subprocess.run(
            ["git", "rev-parse", "--short", "HEAD"],
            capture_output=True, text=True,
        ).stdout.strip() or "not a git repository",
    }


(args.output / "run.json").write_text(json.dumps(run_metadata(args), indent=2))

Six months later, “which threshold produced this table” has an answer that does not depend on anyone’s memory.

Then the check from lesson 2, which is the real test:

rm -rf outputs/
uv run python scripts/indicator_table.py --input data/raw/muac.csv --output outputs/tables
git status

If that does not restore the outputs exactly, something is not reproducible.

Handing it over

The README from lesson 2 needs one more line now — the command with real arguments:

## Run

    uv sync
    uv run python scripts/indicator_table.py \
        --input data/raw/muac-screening-artibonite-2024.v1.csv \
        --output outputs/tables

Thresholds default to MUAC < 125 mm (GAM) and < 115 mm (SAM).
Pass --gam-threshold to change it; the value used is recorded in
outputs/tables/run.json.

Two commands, and the thresholds visible without opening the code. That is the whole handover.

Where this course ends

You can build an environment that reinstalls offline, read any export without corrupting it, turn its codes into values, aggregate to a numerator and a denominator that came from the same operation, get the date arithmetic right, and hand the result to someone else as a script rather than as a favour.

What this course deliberately did not teach is what to compute — which indicator, which denominator, which threshold, and how to defend it. That is Data Analysis Foundations, and the sector courses in Sector Analysis on the programme roadmap take it further into the classifications your cluster holds you to.

If your team works in R rather than Python, R for Programme Data covers the same ground in that language.