{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CMAM programme dashboard — analysis notebook\n",
    "\n",
    "*Nutrition programme dashboard · project deliverable*\n",
    "\n",
    "Cassion · data-analysis.cassion.dev\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thecassion/cassion-learning-platform/blob/main/apps/data-analysis/public/projects/nutrition-programme-dashboard/notebooks/nutrition-dashboard.python.en.ipynb)"
   ],
   "id": "cell-000"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What this notebook is\n",
    "\n",
    "The analysis behind the dashboard: it takes the screening register, produces the\n",
    "three figures the dashboard shows, and states what each one counts. It is a\n",
    "project deliverable rather than a lesson — it assumes the Foundations course and\n",
    "does not re-explain the cleaning.\n",
    "\n",
    "**The decision it informs.** Which of twelve sites to reallocate RUTF stock to\n",
    "before the next quarter opens. That is a ranking problem with a deadline, and\n",
    "the constraint the dashboard exists to respect is that admission figures used to\n",
    "reach the programme manager six weeks after the reporting month.\n",
    "\n",
    "Every dataset on this platform is synthetic. No real child is described, and\n",
    "these figures must not be cited as a real nutrition situation.\n",
    "\n",
    "## Setup"
   ],
   "id": "cell-001"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "URL = (\n",
    "    \"https://data-analysis.cassion.dev/datasets/files/\"\n",
    "    \"muac-screening-artibonite-2024.v1.csv\"\n",
    ")\n",
    "\n",
    "SAM_MM, GAM_MM = 115, 125\n",
    "PLAUSIBLE_MM = (80, 220)\n",
    "\n",
    "muac = pd.read_csv(\n",
    "    URL,\n",
    "    dtype={\"child_id\": \"string\", \"commune\": \"string\", \"sex\": \"string\"},\n",
    "    na_values={\"muac_mm\": [\"-99\"]},\n",
    ")\n",
    "muac[\"screening_date\"] = pd.to_datetime(muac[\"screening_date\"], format=\"%Y-%m-%d\")\n",
    "muac[\"month\"] = muac[\"screening_date\"].dt.to_period(\"M\")\n",
    "\n",
    "len(muac)"
   ],
   "id": "cell-002"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning, applied not explained\n",
    "\n",
    "The decisions and their rationale are in `docs/technical-notes.md`; the code here\n",
    "applies them so the dashboard is reproducible from the raw export."
   ],
   "id": "cell-003"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "unit_error = muac[\"muac_mm\"].notna() & (muac[\"muac_mm\"] < 40)\n",
    "muac.loc[unit_error, \"muac_mm\"] *= 10\n",
    "\n",
    "implausible = muac[\"muac_mm\"].notna() & ~muac[\"muac_mm\"].between(*PLAUSIBLE_MM)\n",
    "muac.loc[implausible, \"muac_mm\"] = np.nan\n",
    "\n",
    "OEDEMA = {\n",
    "    \"true\": True, \"TRUE\": True, \"Y\": True, \"y\": True, \"yes\": True,\n",
    "    \"false\": False, \"FALSE\": False, \"N\": False, \"n\": False, \"no\": False,\n",
    "}\n",
    "muac[\"oedema\"] = muac[\"oedema\"].astype(\"string\").str.strip().map(OEDEMA)\n",
    "\n",
    "before = len(muac)\n",
    "muac = muac.drop_duplicates()\n",
    "\n",
    "cleaning = pd.Series({\n",
    "    \"unit errors corrected\": int(unit_error.sum()),\n",
    "    \"implausible set to missing\": int(implausible.sum()),\n",
    "    \"exact duplicates removed\": before - len(muac),\n",
    "    \"oedema still unrecorded\": int(muac[\"oedema\"].isna().sum()),\n",
    "    \"rows analysed\": len(muac),\n",
    "})\n",
    "cleaning"
   ],
   "id": "cell-004"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Panel 1 — caseload, and how current it is\n",
    "\n",
    "The dashboard's first number is not a rate. It is how many children were\n",
    "screened and how recently, because a prevalence figure computed on a month that\n",
    "is only half reported is the failure this project exists to prevent."
   ],
   "id": "cell-005"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "muac[\"assessed\"] = muac[\"muac_mm\"].notna() | muac[\"oedema\"].notna()\n",
    "muac[\"gam\"] = np.where(\n",
    "    ~muac[\"assessed\"], np.nan,\n",
    "    ((muac[\"muac_mm\"] < GAM_MM) | (muac[\"oedema\"] == True)).astype(float),\n",
    ")\n",
    "muac[\"sam\"] = np.where(\n",
    "    ~muac[\"assessed\"], np.nan,\n",
    "    ((muac[\"muac_mm\"] < SAM_MM) | (muac[\"oedema\"] == True)).astype(float),\n",
    ")\n",
    "\n",
    "monthly = muac.groupby(\"month\").agg(\n",
    "    screened=(\"child_id\", \"size\"),\n",
    "    assessed=(\"assessed\", \"sum\"),\n",
    "    sam_cases=(\"sam\", \"sum\"),\n",
    ")\n",
    "monthly[\"assessment_rate\"] = (monthly[\"assessed\"] / monthly[\"screened\"]).round(3)\n",
    "monthly.tail(6)"
   ],
   "id": "cell-006"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first and last months are short — screening runs from mid-January to\n",
    "mid-December — so the dashboard labels them rather than letting a reader take a\n",
    "partial month for a decline."
   ],
   "id": "cell-007"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"first screening:\", muac[\"screening_date\"].min().date())\n",
    "print(\"last screening :\", muac[\"screening_date\"].max().date())"
   ],
   "id": "cell-008"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Panel 2 — the site ranking, with its uncertainty"
   ],
   "id": "cell-009"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import beta\n",
    "\n",
    "def interval(cases, n, confidence=0.95):\n",
    "    if n == 0:\n",
    "        return (np.nan, np.nan)\n",
    "    a = 1 - confidence\n",
    "    low = beta.ppf(a / 2, cases, n - cases + 1) if cases > 0 else 0.0\n",
    "    high = beta.ppf(1 - a / 2, cases + 1, n - cases) if cases < n else 1.0\n",
    "    return (low, high)\n",
    "\n",
    "sites = muac.groupby(\"commune\").agg(\n",
    "    screened=(\"child_id\", \"size\"),\n",
    "    assessed=(\"assessed\", \"sum\"),\n",
    "    gam_cases=(\"gam\", \"sum\"),\n",
    "    sam_cases=(\"sam\", \"sum\"),\n",
    ")\n",
    "sites[\"gam_rate\"] = sites[\"gam_cases\"] / sites[\"assessed\"]\n",
    "sites[\"sam_rate\"] = sites[\"sam_cases\"] / sites[\"assessed\"]\n",
    "\n",
    "bounds = sites.apply(lambda r: interval(r[\"gam_cases\"], r[\"assessed\"]), axis=1)\n",
    "sites[\"gam_low\"] = [b[0] for b in bounds]\n",
    "sites[\"gam_high\"] = [b[1] for b in bounds]\n",
    "\n",
    "sites = sites.sort_values(\"gam_rate\", ascending=False)\n",
    "sites.round(3)"
   ],
   "id": "cell-010"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The ranking is not a queue.** Several intervals overlap, so the order between\n",
    "those sites is not supported by the screening. The dashboard renders the\n",
    "interval, not just the point, precisely so a manager allocating stock can see\n",
    "which gaps are real."
   ],
   "id": "cell-011"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EMERGENCY = 0.15\n",
    "\n",
    "decisive = sites[sites[\"gam_low\"] > sites[\"gam_rate\"].median()]\n",
    "print(f\"sites clearly above the district median: {list(decisive.index)}\")\n",
    "print(f\"sites whose interval crosses the {EMERGENCY:.0%} emergency threshold: \"\n",
    "      f\"{list(sites[(sites['gam_high'] > EMERGENCY)].index)}\")"
   ],
   "id": "cell-012"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Panel 3 — where the data is too thin to act on\n",
    "\n",
    "The panel that makes the dashboard trustworthy rather than impressive. A site\n",
    "with poor assessment coverage produces a rate the manager should not act on, and\n",
    "saying so is more useful than a confident number."
   ],
   "id": "cell-013"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "coverage = muac.groupby(\"commune\").agg(\n",
    "    screened=(\"child_id\", \"size\"),\n",
    "    assessment_rate=(\"assessed\", \"mean\"),\n",
    "    missing_age=(\"age_months\", lambda s: s.isna().mean()),\n",
    ").round(3)\n",
    "\n",
    "coverage[\"interval_width\"] = (sites[\"gam_high\"] - sites[\"gam_low\"]).round(3)\n",
    "coverage.sort_values(\"interval_width\", ascending=False)"
   ],
   "id": "cell-014"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One site's missing-age rate stands well clear of the rest — a single campaign\n",
    "week with a misconfigured form. It does not affect the MUAC indicator, which\n",
    "needs no age, and the dashboard says so rather than dropping the site.\n",
    "\n",
    "## The three figures the dashboard shows"
   ],
   "id": "cell-015"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "district_assessed = int(sites[\"assessed\"].sum())\n",
    "district_gam = sites[\"gam_cases\"].sum() / district_assessed\n",
    "district_sam = sites[\"sam_cases\"].sum() / district_assessed\n",
    "low, high = interval(sites[\"gam_cases\"].sum(), district_assessed)\n",
    "\n",
    "summary = pd.Series({\n",
    "    \"children assessed\": district_assessed,\n",
    "    \"GAM\": f\"{district_gam:.1%} ({low:.1%} - {high:.1%})\",\n",
    "    \"SAM\": f\"{district_sam:.1%}\",\n",
    "    \"sites above district median, decisively\": len(decisive),\n",
    "})\n",
    "summary"
   ],
   "id": "cell-016"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Indicator definitions, printed with the output\n",
    "\n",
    "The dashboard carries this panel too. A number and its definition travel\n",
    "together, or the monthly argument about the denominator starts again."
   ],
   "id": "cell-017"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "definitions = pd.DataFrame([\n",
    "    (\"GAM\", \"MUAC < 125 mm or bilateral pitting oedema\",\n",
    "     \"children with a MUAC measurement or a recorded oedema assessment\"),\n",
    "    (\"SAM\", \"MUAC < 115 mm or bilateral pitting oedema\",\n",
    "     \"children with a MUAC measurement or a recorded oedema assessment\"),\n",
    "    (\"Assessment rate\", \"children with MUAC or oedema recorded\",\n",
    "     \"children screened\"),\n",
    "], columns=[\"indicator\", \"numerator\", \"denominator\"])\n",
    "definitions"
   ],
   "id": "cell-018"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What none of these is.** Programme coverage. This register holds the children\n",
    "who came to be screened, not the children who exist, so nothing here estimates\n",
    "what share of cases the programme reaches. That requires a population figure and\n",
    "a coverage survey, and the report says so in its limitations."
   ],
   "id": "cell-019"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "file_extension": ".py"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}