{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Coverage and dropout rate by facility\n",
    "\n",
    "*Routine vaccination coverage, 2024*\n",
    "\n",
    "Cassion · data-analysis.cassion.dev\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thecassion/cassion-learning-platform/blob/main/apps/data-analysis/public/datasets/examples/routine-vaccination-coverage-2024/coverage-dropout.python.en.ipynb)"
   ],
   "id": "cell-000"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What this produces\n",
    "\n",
    "Monthly and annual coverage by antigen, penta1-to-penta3 dropout by facility, and\n",
    "the over-reporting flag — applied at an aggregation level where it actually\n",
    "means something.\n",
    "\n",
    "Every dataset on this platform is synthetic. Coverage here describes no real\n",
    "district.\n",
    "\n",
    "## Setup"
   ],
   "id": "cell-001"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "URL = (\n",
    "    \"https://data-analysis.cassion.dev/datasets/files/\"\n",
    "    \"vaccination-coverage-2024.v1.csv\"\n",
    ")\n",
    "\n",
    "epi = pd.read_csv(URL, dtype={\"facility_id\": \"string\"})\n",
    "epi[\"period\"] = pd.to_datetime(epi[\"period\"])\n",
    "epi[\"month\"] = epi[\"period\"].dt.to_period(\"M\")\n",
    "\n",
    "print(f\"{epi['facility_id'].nunique()} facilities, \"\n",
    "      f\"{epi['month'].nunique()} months, \"\n",
    "      f\"{epi['antigen'].nunique()} antigens\")\n",
    "epi.head()"
   ],
   "id": "cell-002"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A non-report is not zero children vaccinated\n",
    "\n",
    "A facility that did not report appears as a row with `report_submitted` false and\n",
    "zero doses — **not** as a missing row. Sum doses without filtering on that flag\n",
    "and you have asserted that no child in that catchment was vaccinated that month."
   ],
   "id": "cell-003"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "not_submitted = ~epi[\"report_submitted\"]\n",
    "print(f\"facility-months not reported : {int(not_submitted.sum() / epi['antigen'].nunique())}\")\n",
    "print(f\"non-reporting rows with 0 doses: \"\n",
    "      f\"{int((not_submitted & (epi['doses_administered'] == 0)).sum())} \"\n",
    "      f\"of {int(not_submitted.sum())}\")"
   ],
   "id": "cell-004"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every one of them. That is the trap: the zero looks like data.\n",
    "\n",
    "## The denominator is annual, the reporting is monthly\n",
    "\n",
    "`target_population` is the **annual** target, repeated on every row. Monthly\n",
    "coverage is doses divided by the target over twelve. Dividing by the full annual\n",
    "figure understates coverage twelvefold, and it is the most common mistake made\n",
    "against data shaped like this."
   ],
   "id": "cell-005"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MONTHS_IN_YEAR = 12\n",
    "\n",
    "penta3 = epi[epi[\"antigen\"] == \"penta3\"].copy()\n",
    "\n",
    "monthly = penta3.groupby(\"month\").apply(\n",
    "    lambda g: pd.Series({\n",
    "        \"doses\": g[\"doses_administered\"].sum(),\n",
    "        \"annual_target\": g[\"target_population\"].sum(),\n",
    "        \"completeness\": g[\"report_submitted\"].mean(),\n",
    "    }),\n",
    "    include_groups=False,\n",
    ")\n",
    "\n",
    "monthly[\"wrong_annual_denominator\"] = monthly[\"doses\"] / monthly[\"annual_target\"]\n",
    "monthly[\"correct_monthly_denominator\"] = monthly[\"doses\"] / (\n",
    "    monthly[\"annual_target\"] / MONTHS_IN_YEAR\n",
    ")\n",
    "monthly.round(3)"
   ],
   "id": "cell-006"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column is a coverage figure of about 5%, which would read as a\n",
    "catastrophic programme failure. It is an arithmetic error.\n",
    "\n",
    "## Dropout between penta1 and penta3\n",
    "\n",
    "Dropout is the share of children who start the series and do not finish it:\n",
    "`(penta1 − penta3) / penta1`. It is a better programme signal than coverage\n",
    "because it does not depend on the population estimate at all — both terms come\n",
    "from the same register."
   ],
   "id": "cell-007"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "reported = epi[epi[\"report_submitted\"]]\n",
    "\n",
    "series = (\n",
    "    reported[reported[\"antigen\"].isin([\"penta1\", \"penta3\"])]\n",
    "    .pivot_table(\n",
    "        index=\"facility_id\", columns=\"antigen\",\n",
    "        values=\"doses_administered\", aggfunc=\"sum\",\n",
    "    )\n",
    "    .dropna()\n",
    ")\n",
    "\n",
    "series[\"dropout\"] = (series[\"penta1\"] - series[\"penta3\"]) / series[\"penta1\"]\n",
    "series = series.sort_values(\"dropout\", ascending=False)\n",
    "\n",
    "print(f\"median dropout: {series['dropout'].median():.1%}\")\n",
    "series.head(10).round(3)"
   ],
   "id": "cell-008"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Median dropout is about 14%. A facility reporting near-zero dropout deserves as\n",
    "much suspicion as one reporting a very high figure — near-zero usually means the\n",
    "penta3 count was reconstructed from the penta1 count rather than counted.\n",
    "\n",
    "## The over-reporting flag, applied at the right level\n",
    "\n",
    "Penta3 exceeding penta1 is impossible in a real schedule: a child cannot receive\n",
    "the third dose without the first. It is the classic over-reporting signal.\n",
    "\n",
    "Applied month by month, it is useless here:"
   ],
   "id": "cell-009"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "by_month = (\n",
    "    reported[reported[\"antigen\"].isin([\"penta1\", \"penta3\"])]\n",
    "    .pivot_table(\n",
    "        index=[\"facility_id\", \"month\"], columns=\"antigen\",\n",
    "        values=\"doses_administered\",\n",
    "    )\n",
    "    .dropna()\n",
    ")\n",
    "by_month[\"impossible\"] = by_month[\"penta3\"] > by_month[\"penta1\"]\n",
    "\n",
    "flagged_once = by_month.groupby(\"facility_id\")[\"impossible\"].any().sum()\n",
    "print(f\"facilities flagged at least once: {flagged_once} \"\n",
    "      f\"of {epi['facility_id'].nunique()}\")"
   ],
   "id": "cell-010"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thirty-seven of thirty-eight. A check that flags almost the whole district\n",
    "identifies nothing — ordinary month-to-month noise crosses that line constantly,\n",
    "because the two doses are given to different children in different months and\n",
    "the counts do not have to move together.\n",
    "\n",
    "The flag has to be applied where the noise averages out:"
   ],
   "id": "cell-011"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "persistent = series[series[\"dropout\"] < 0]\n",
    "print(f\"facilities with penta3 above penta1 on the annual total: {len(persistent)}\")\n",
    "persistent.round(3)"
   ],
   "id": "cell-012"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Six facilities. That is a list a supervisor can act on."
   ],
   "id": "cell-013"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "months_flagged = by_month.groupby(\"facility_id\")[\"impossible\"].agg([\"sum\", \"size\"])\n",
    "months_flagged.columns = [\"months flagged\", \"months reported\"]\n",
    "months_flagged.loc[persistent.index].sort_values(\"months flagged\", ascending=False)"
   ],
   "id": "cell-014"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Coverage by antigen, on a denominator that holds"
   ],
   "id": "cell-015"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def annual_coverage(df):\n",
    "    reported_only = df[df[\"report_submitted\"]]\n",
    "    return (\n",
    "        reported_only.groupby(\"antigen\")\n",
    "        .apply(\n",
    "            lambda g: g[\"doses_administered\"].sum()\n",
    "            / (g[\"target_population\"].sum() / MONTHS_IN_YEAR),\n",
    "            include_groups=False,\n",
    "        )\n",
    "        .rename(\"coverage\")\n",
    "        .to_frame()\n",
    "        .round(3)\n",
    "    )\n",
    "\n",
    "annual_coverage(epi)"
   ],
   "id": "cell-016"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The denominator here counts only facilities that reported, which keeps the\n",
    "numerator and the denominator describing the same set of facilities. That is the\n",
    "subject of the completeness-adjustment example, and it is the difference between\n",
    "a coverage figure that moves with reporting and one that moves with vaccination.\n",
    "\n",
    "## What to report\n",
    "\n",
    "Coverage with its denominator stated, dropout as the programme signal that does\n",
    "not depend on a population estimate, and the over-reporting list with the\n",
    "aggregation level it was computed at. A flag that fires on thirty-seven of\n",
    "thirty-eight facilities is not a finding — it is a badly specified check."
   ],
   "id": "cell-017"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "file_extension": ".py"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}