{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adjusting coverage for reporting completeness\n",
    "\n",
    "*Routine vaccination coverage, 2024 · R*\n",
    "\n",
    "Cassion · data-analysis.cassion.dev\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thecassion/cassion-learning-platform/blob/main/apps/data-analysis/public/datasets/examples/routine-vaccination-coverage-2024/completeness-adjustment.r.en.ipynb)"
   ],
   "id": "cell-000"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The problem in one sentence\n",
    "\n",
    "In August, reported penta3 coverage falls to under a third. Nothing happened to\n",
    "vaccination — 71% of facilities simply did not send a report, and their zeros\n",
    "went into the numerator while their catchment populations stayed in the\n",
    "denominator.\n",
    "\n",
    "Every dataset on this platform is synthetic. Coverage here describes no real\n",
    "district.\n",
    "\n",
    "## Setup"
   ],
   "id": "cell-001"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| message: false\n",
    "library(readr)\n",
    "library(dplyr)\n",
    "library(tidyr)\n",
    "library(ggplot2)\n",
    "\n",
    "URL <- paste0(\n",
    "  \"https://data-analysis.cassion.dev/datasets/files/\",\n",
    "  \"vaccination-coverage-2024.v1.csv\"\n",
    ")\n",
    "\n",
    "epi <- read_csv(URL, col_types = cols(\n",
    "  facility_id = col_character(),\n",
    "  period      = col_date(),\n",
    "  .default    = col_guess()\n",
    ")) |>\n",
    "  mutate(month = format(period, \"%Y-%m\"))\n",
    "\n",
    "MONTHS_IN_YEAR <- 12"
   ],
   "id": "cell-002"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Completeness is an indicator in its own right\n",
    "\n",
    "Report it before you report anything computed from the data it gates. A coverage\n",
    "figure standing next to a 29% completeness figure is read very differently from\n",
    "the same figure standing alone."
   ],
   "id": "cell-003"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "completeness <- epi |>\n",
    "  distinct(facility_id, month, report_submitted) |>\n",
    "  group_by(month) |>\n",
    "  summarise(\n",
    "    facilities   = n(),\n",
    "    reported     = sum(report_submitted),\n",
    "    completeness = round(mean(report_submitted), 3),\n",
    "    .groups = \"drop\"\n",
    "  )\n",
    "\n",
    "completeness"
   ],
   "id": "cell-004"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "August at 29% and September at 45%. Everything below is about what those two\n",
    "months do to a coverage series.\n",
    "\n",
    "## Three denominators, three different stories"
   ],
   "id": "cell-005"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "penta3 <- epi |> filter(antigen == \"penta3\")\n",
    "\n",
    "series <- penta3 |>\n",
    "  group_by(month) |>\n",
    "  summarise(\n",
    "    doses            = sum(doses_administered),\n",
    "    all_target       = sum(target_population),\n",
    "    reported_target  = sum(target_population[report_submitted]),\n",
    "    completeness     = mean(report_submitted),\n",
    "    .groups = \"drop\"\n",
    "  ) |>\n",
    "  mutate(\n",
    "    uncorrected  = doses / (all_target / MONTHS_IN_YEAR),\n",
    "    reported_only = doses / (reported_target / MONTHS_IN_YEAR),\n",
    "    scaled       = uncorrected / completeness\n",
    "  )\n",
    "\n",
    "series |>\n",
    "  select(month, completeness, uncorrected, reported_only, scaled) |>\n",
    "  mutate(across(where(is.numeric), ~ round(.x, 3)))"
   ],
   "id": "cell-006"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Uncorrected** counts every facility's catchment in the denominator and only the\n",
    "reporting facilities' doses in the numerator. The two do not describe the same\n",
    "population, and the result tracks reporting rather than vaccination.\n",
    "\n",
    "**Reported-only** restricts the denominator to the facilities that reported.\n",
    "Numerator and denominator now describe the same set of facilities, and the series\n",
    "is stable across August.\n",
    "\n",
    "**Scaled** divides the uncorrected figure by completeness. It is the adjustment\n",
    "people reach for first, and it is the one to be careful with.\n",
    "\n",
    "## Why scaling by completeness over-corrects"
   ],
   "id": "cell-007"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "series |>\n",
    "  select(month, completeness, reported_only, scaled) |>\n",
    "  mutate(\n",
    "    difference = round(scaled - reported_only, 3),\n",
    "    across(where(is.numeric), ~ round(.x, 3))\n",
    "  ) |>\n",
    "  arrange(desc(abs(difference)))"
   ],
   "id": "cell-008"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In August, scaling produces coverage near 99% — higher than any month when\n",
    "almost everyone reported. Dividing by completeness assumes the silent facilities\n",
    "would have performed exactly like the reporting ones. They usually would not:\n",
    "facilities that stop reporting are often the ones with a stockout, a staffing\n",
    "gap or an access problem, which is to say the ones performing worst.\n",
    "\n",
    "The assumption is not always wrong, but it must be stated, and it must not be\n",
    "made silently by an analyst reaching for the simplest formula.\n",
    "\n",
    "## The plot that makes the point"
   ],
   "id": "cell-009"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| fig-width: 8\n",
    "#| fig-height: 5\n",
    "series |>\n",
    "  select(month, uncorrected, reported_only, scaled) |>\n",
    "  pivot_longer(-month, names_to = \"method\", values_to = \"coverage\") |>\n",
    "  ggplot(aes(x = month, y = 100 * coverage, group = method, colour = method)) +\n",
    "  geom_line(linewidth = 0.8) +\n",
    "  geom_point(size = 1.8) +\n",
    "  scale_colour_manual(values = c(\n",
    "    uncorrected   = \"#B5533C\",\n",
    "    reported_only = \"#2F5D50\",\n",
    "    scaled        = \"#9AA8A3\"\n",
    "  )) +\n",
    "  labs(\n",
    "    x = NULL, y = \"penta3 coverage (%)\", colour = NULL,\n",
    "    title = \"The August cliff is a reporting artefact, not a programme collapse\"\n",
    "  ) +\n",
    "  theme_minimal(base_size = 11) +\n",
    "  theme(\n",
    "    axis.text.x = element_text(angle = 45, hjust = 1),\n",
    "    panel.grid.minor = element_blank()\n",
    "  )"
   ],
   "id": "cell-010"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The uncorrected line has a cliff in August. The reported-only line does not. The\n",
    "cliff is a property of who sent a form, and a programme manager shown the first\n",
    "chart would spend a month investigating a collapse that did not happen.\n",
    "\n",
    "## Which facilities went quiet"
   ],
   "id": "cell-011"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "silent <- epi |>\n",
    "  distinct(facility_id, month, report_submitted, facility_type) |>\n",
    "  filter(month %in% c(\"2024-08\", \"2024-09\")) |>\n",
    "  group_by(facility_type) |>\n",
    "  summarise(\n",
    "    facility_months = n(),\n",
    "    reported        = sum(report_submitted),\n",
    "    completeness    = round(mean(report_submitted), 3),\n",
    "    .groups = \"drop\"\n",
    "  )\n",
    "\n",
    "silent"
   ],
   "id": "cell-012"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If completeness differs by facility type, the reported-only denominator is still\n",
    "biased — it now describes the facilities that report, which are not a random\n",
    "sample of all facilities. Check this before presenting the corrected series as\n",
    "though it were the truth.\n",
    "\n",
    "## What to report\n",
    "\n",
    "Completeness beside coverage, always, in the same table. The denominator you\n",
    "used, named. And if you scaled by completeness, the assumption that the silent\n",
    "facilities perform like the reporting ones, written out — because that assumption\n",
    "is doing more work than the arithmetic is."
   ],
   "id": "cell-013"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "name": "R",
   "file_extension": ".r"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}