{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Referral completion and where the pathway breaks\n",
    "\n",
    "*Protection referrals, 2024*\n",
    "\n",
    "Cassion · data-analysis.cassion.dev\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thecassion/cassion-learning-platform/blob/main/apps/data-analysis/public/datasets/examples/protection-referrals-2024/referral-completion.python.en.ipynb)"
   ],
   "id": "cell-000"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before any code\n",
    "\n",
    "This dataset models protection and GBV case data. It is synthetic — no real\n",
    "person is described — and it must never be used as a template for storing real\n",
    "case data. The safe version of that is a consent-governed case management\n",
    "system, not a CSV.\n",
    "\n",
    "Notice what is **not** here: no names, no contact details, no free text, no\n",
    "incident date, no location below admin2, no exact age, no incident type, no\n",
    "perpetrator detail. None of those are needed to measure whether a referral\n",
    "pathway works, and under GBV information management principles the\n",
    "incident-level fields are never shared outside the case management agency at\n",
    "all. Collecting less than you could is the discipline being modelled.\n",
    "\n",
    "## Setup"
   ],
   "id": "cell-001"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "URL = (\n",
    "    \"https://data-analysis.cassion.dev/datasets/files/\"\n",
    "    \"protection-referrals-2024.v1.csv\"\n",
    ")\n",
    "\n",
    "cases = pd.read_csv(URL, dtype={\"case_id\": \"string\"})\n",
    "print(cases.shape)\n",
    "cases.head()"
   ],
   "id": "cell-002"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Consent gates the denominator\n",
    "\n",
    "**The completion denominator is cases that consented to referral, not all\n",
    "cases.** Counting a non-consenting case as a pathway failure both misstates\n",
    "performance and misrepresents a person's decision — the pathway did exactly what\n",
    "it should when someone declined."
   ],
   "id": "cell-003"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"cases                    : {len(cases)}\")\n",
    "print(f\"consented to referral    : {cases['consent_to_refer'].sum()} \"\n",
    "      f\"({cases['consent_to_refer'].mean():.1%})\")\n",
    "\n",
    "consenting = cases[cases[\"consent_to_refer\"]].copy()\n",
    "print(f\"reached a service        : {consenting['referral_accepted'].sum()} \"\n",
    "      f\"({consenting['referral_accepted'].mean():.1%})\")"
   ],
   "id": "cell-004"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Two figures, two different meanings. The 88.5% is a measure of how the service\n",
    "was offered — whether people felt safe enough to accept a referral. The 46% is\n",
    "the pathway. Reporting only the second, against the full caseload, would give a\n",
    "completion rate of about 41% and blame the pathway for the 11.5% who declined.\n",
    "\n",
    "## Resolve the contradictions before trusting anything\n",
    "\n",
    "Eleven cases record a time to first service even though the referral was never\n",
    "accepted. Six of those show no referral made at all. These are logical\n",
    "impossibilities, and a completion rate computed over them is computed over\n",
    "records that cannot all be right."
   ],
   "id": "cell-005"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "recorded_time = cases[\"days_to_first_service\"].notna()\n",
    "\n",
    "contradictions = pd.DataFrame({\n",
    "    \"time recorded, not accepted\": [\n",
    "        int((~cases[\"referral_accepted\"] & recorded_time).sum())\n",
    "    ],\n",
    "    \"time recorded, no referral made\": [\n",
    "        int((~cases[\"referral_made\"] & recorded_time).sum())\n",
    "    ],\n",
    "    \"time recorded, no consent\": [\n",
    "        int((~cases[\"consent_to_refer\"] & recorded_time).sum())\n",
    "    ],\n",
    "    \"accepted, no time recorded\": [\n",
    "        int((cases[\"referral_accepted\"] & ~recorded_time).sum())\n",
    "    ],\n",
    "}).T\n",
    "contradictions.columns = [\"cases\"]\n",
    "contradictions"
   ],
   "id": "cell-006"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last row matters as much as the others: forty accepted referrals have no time\n",
    "recorded, so **the timeliness denominator is smaller than the completion\n",
    "denominator.** Using one for both misstates both."
   ],
   "id": "cell-007"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases[\"contradictory\"] = (~cases[\"referral_accepted\"]) & recorded_time\n",
    "print(f\"flagged as contradictory: {int(cases['contradictory'].sum())}\")"
   ],
   "id": "cell-008"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Flag them rather than dropping them. In a case management context a\n",
    "contradictory record is a data entry issue to send back to the caseworker, and\n",
    "deleting it destroys the only trace that the case existed.\n",
    "\n",
    "## Normalise the disability field before disaggregating on it\n",
    "\n",
    "One area recorded disability as `Yes` and `No` rather than `true` and `false`.\n",
    "Left alone, the disaggregation fragments into four categories, two of them too\n",
    "small to interpret — and those two come from one area, so they are not a random\n",
    "subset."
   ],
   "id": "cell-009"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(cases[\"disability_reported\"].value_counts(dropna=False))"
   ],
   "id": "cell-010"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DISABILITY = {\"true\": True, \"yes\": True, \"false\": False, \"no\": False}\n",
    "cases[\"disability\"] = (\n",
    "    cases[\"disability_reported\"].astype(\"string\").str.strip().str.lower().map(DISABILITY)\n",
    ")\n",
    "consenting = cases[cases[\"consent_to_refer\"]].copy()\n",
    "print(consenting[\"disability\"].value_counts(dropna=False))"
   ],
   "id": "cell-011"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Where the pathway breaks"
   ],
   "id": "cell-012"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def completion(df, by):\n",
    "    out = df.groupby(by).agg(\n",
    "        cases=(\"referral_accepted\", \"size\"),\n",
    "        completed=(\"referral_accepted\", \"sum\"),\n",
    "    )\n",
    "    out[\"completion\"] = (out[\"completed\"] / out[\"cases\"]).round(3)\n",
    "    return out.sort_values(\"completion\")\n",
    "\n",
    "completion(consenting, \"service_requested\")"
   ],
   "id": "cell-013"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Livelihood support completes at about 23% against health at about 62%. That is\n",
    "not a caseworker performance gap — it is a statement about which services exist\n",
    "and have capacity. A pathway analysis that stops at an overall 46% hides the\n",
    "entire finding."
   ],
   "id": "cell-014"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "completion(consenting, \"admin2\")"
   ],
   "id": "cell-015"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Between about 31% and 57% across the six areas. Combine the two cuts and the\n",
    "failing node becomes locatable:"
   ],
   "id": "cell-016"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid = pd.crosstab(\n",
    "    consenting[\"admin2\"],\n",
    "    consenting[\"service_requested\"],\n",
    "    values=consenting[\"referral_accepted\"],\n",
    "    aggfunc=\"mean\",\n",
    ").round(2)\n",
    "\n",
    "counts = pd.crosstab(consenting[\"admin2\"], consenting[\"service_requested\"])\n",
    "grid.where(counts >= 20)"
   ],
   "id": "cell-017"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cells with fewer than twenty cases are blanked rather than shown. A completion\n",
    "rate on eight cases is not a finding, and in protection work a small cell is also\n",
    "a disclosure risk — see below.\n",
    "\n",
    "## The equity finding"
   ],
   "id": "cell-018"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "by_disability = completion(consenting.dropna(subset=[\"disability\"]), \"disability\")\n",
    "by_disability"
   ],
   "id": "cell-019"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cases where a disability was reported complete at about 31% against 48% where\n",
    "none was reported. This is the finding the dataset exists to surface, and it is\n",
    "the kind that only appears if you disaggregate."
   ],
   "id": "cell-020"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.stats import chi2_contingency\n",
    "\n",
    "table = pd.crosstab(\n",
    "    consenting.dropna(subset=[\"disability\"])[\"disability\"],\n",
    "    consenting.dropna(subset=[\"disability\"])[\"referral_accepted\"],\n",
    ")\n",
    "chi2, p, dof, expected = chi2_contingency(table)\n",
    "print(f\"difference in completion: \"\n",
    "      f\"{by_disability['completion'].iloc[-1] - by_disability['completion'].iloc[0]:+.3f}\")\n",
    "print(f\"chi-square p            : {p:.2e}\")"
   ],
   "id": "cell-021"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A seventeen-point gap, and unlikely to be chance. What it does not tell you is\n",
    "*why* — whether services are physically inaccessible, whether referral pathways\n",
    "assume a mobility that some clients do not have, or whether something else\n",
    "entirely. That question is answered by asking caseworkers, not by this table.\n",
    "\n",
    "## Small cells are a protection risk, not just a statistical one"
   ],
   "id": "cell-022"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "risky = pd.crosstab(consenting[\"admin2\"], consenting[\"case_category\"])\n",
    "risky.where(risky < 20)"
   ],
   "id": "cell-023"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every cell here clears twenty, so nothing is suppressed — and that empty result\n",
    "is exactly why the check belongs in the pipeline permanently rather than being\n",
    "run once. In a district table, a cell of three GBV cases can identify a survivor\n",
    "to anyone who knows the area. Treat the suppression threshold as a protection\n",
    "decision made with the case management agency, not as a formatting preference.\n",
    "\n",
    "## What to report\n",
    "\n",
    "Completion against the consent-gated denominator, decomposed far enough to locate\n",
    "the failing node, with contradictory records flagged and counted. The disability\n",
    "gap, stated plainly. And nothing at a granularity that could identify a person —\n",
    "which in this sector is the constraint that outranks every analytical preference."
   ],
   "id": "cell-024"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "file_extension": ".py"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}