---
title: "Referral completion and where the pathway breaks"
subtitle: "Protection referrals, 2024"
author: "Cassion · data-analysis.cassion.dev"
format:
  html:
    toc: true
    code-fold: false
jupyter: python3
---

## Before any code

This dataset models protection and GBV case data. It is synthetic — no real
person is described — and it must never be used as a template for storing real
case data. The safe version of that is a consent-governed case management
system, not a CSV.

Notice what is **not** here: no names, no contact details, no free text, no
incident date, no location below admin2, no exact age, no incident type, no
perpetrator detail. None of those are needed to measure whether a referral
pathway works, and under GBV information management principles the
incident-level fields are never shared outside the case management agency at
all. Collecting less than you could is the discipline being modelled.

## Setup

```{python}
import pandas as pd
import numpy as np

URL = (
    "https://data-analysis.cassion.dev/datasets/files/"
    "protection-referrals-2024.v1.csv"
)

cases = pd.read_csv(URL, dtype={"case_id": "string"})
print(cases.shape)
cases.head()
```

## Consent gates the denominator

**The completion denominator is cases that consented to referral, not all
cases.** Counting a non-consenting case as a pathway failure both misstates
performance and misrepresents a person's decision — the pathway did exactly what
it should when someone declined.

```{python}
print(f"cases                    : {len(cases)}")
print(f"consented to referral    : {cases['consent_to_refer'].sum()} "
      f"({cases['consent_to_refer'].mean():.1%})")

consenting = cases[cases["consent_to_refer"]].copy()
print(f"reached a service        : {consenting['referral_accepted'].sum()} "
      f"({consenting['referral_accepted'].mean():.1%})")
```

Two figures, two different meanings. The 88.5% is a measure of how the service
was offered — whether people felt safe enough to accept a referral. The 46% is
the pathway. Reporting only the second, against the full caseload, would give a
completion rate of about 41% and blame the pathway for the 11.5% who declined.

## Resolve the contradictions before trusting anything

Eleven cases record a time to first service even though the referral was never
accepted. Six of those show no referral made at all. These are logical
impossibilities, and a completion rate computed over them is computed over
records that cannot all be right.

```{python}
recorded_time = cases["days_to_first_service"].notna()

contradictions = pd.DataFrame({
    "time recorded, not accepted": [
        int((~cases["referral_accepted"] & recorded_time).sum())
    ],
    "time recorded, no referral made": [
        int((~cases["referral_made"] & recorded_time).sum())
    ],
    "time recorded, no consent": [
        int((~cases["consent_to_refer"] & recorded_time).sum())
    ],
    "accepted, no time recorded": [
        int((cases["referral_accepted"] & ~recorded_time).sum())
    ],
}).T
contradictions.columns = ["cases"]
contradictions
```

The last row matters as much as the others: forty accepted referrals have no time
recorded, so **the timeliness denominator is smaller than the completion
denominator.** Using one for both misstates both.

```{python}
cases["contradictory"] = (~cases["referral_accepted"]) & recorded_time
print(f"flagged as contradictory: {int(cases['contradictory'].sum())}")
```

Flag them rather than dropping them. In a case management context a
contradictory record is a data entry issue to send back to the caseworker, and
deleting it destroys the only trace that the case existed.

## Normalise the disability field before disaggregating on it

One area recorded disability as `Yes` and `No` rather than `true` and `false`.
Left alone, the disaggregation fragments into four categories, two of them too
small to interpret — and those two come from one area, so they are not a random
subset.

```{python}
print(cases["disability_reported"].value_counts(dropna=False))
```

```{python}
DISABILITY = {"true": True, "yes": True, "false": False, "no": False}
cases["disability"] = (
    cases["disability_reported"].astype("string").str.strip().str.lower().map(DISABILITY)
)
consenting = cases[cases["consent_to_refer"]].copy()
print(consenting["disability"].value_counts(dropna=False))
```

## Where the pathway breaks

```{python}
def completion(df, by):
    out = df.groupby(by).agg(
        cases=("referral_accepted", "size"),
        completed=("referral_accepted", "sum"),
    )
    out["completion"] = (out["completed"] / out["cases"]).round(3)
    return out.sort_values("completion")

completion(consenting, "service_requested")
```

Livelihood support completes at about 23% against health at about 62%. That is
not a caseworker performance gap — it is a statement about which services exist
and have capacity. A pathway analysis that stops at an overall 46% hides the
entire finding.

```{python}
completion(consenting, "admin2")
```

Between about 31% and 57% across the six areas. Combine the two cuts and the
failing node becomes locatable:

```{python}
grid = pd.crosstab(
    consenting["admin2"],
    consenting["service_requested"],
    values=consenting["referral_accepted"],
    aggfunc="mean",
).round(2)

counts = pd.crosstab(consenting["admin2"], consenting["service_requested"])
grid.where(counts >= 20)
```

Cells with fewer than twenty cases are blanked rather than shown. A completion
rate on eight cases is not a finding, and in protection work a small cell is also
a disclosure risk — see below.

## The equity finding

```{python}
by_disability = completion(consenting.dropna(subset=["disability"]), "disability")
by_disability
```

Cases where a disability was reported complete at about 31% against 48% where
none was reported. This is the finding the dataset exists to surface, and it is
the kind that only appears if you disaggregate.

```{python}
from scipy.stats import chi2_contingency

table = pd.crosstab(
    consenting.dropna(subset=["disability"])["disability"],
    consenting.dropna(subset=["disability"])["referral_accepted"],
)
chi2, p, dof, expected = chi2_contingency(table)
print(f"difference in completion: "
      f"{by_disability['completion'].iloc[-1] - by_disability['completion'].iloc[0]:+.3f}")
print(f"chi-square p            : {p:.2e}")
```

A seventeen-point gap, and unlikely to be chance. What it does not tell you is
*why* — whether services are physically inaccessible, whether referral pathways
assume a mobility that some clients do not have, or whether something else
entirely. That question is answered by asking caseworkers, not by this table.

## Small cells are a protection risk, not just a statistical one

```{python}
risky = pd.crosstab(consenting["admin2"], consenting["case_category"])
risky.where(risky < 20)
```

Every cell here clears twenty, so nothing is suppressed — and that empty result
is exactly why the check belongs in the pipeline permanently rather than being
run once. In a district table, a cell of three GBV cases can identify a survivor
to anyone who knows the area. Treat the suppression threshold as a protection
decision made with the case management agency, not as a formatting preference.

## What to report

Completion against the consent-gated denominator, decomposed far enough to locate
the failing node, with contradictory records flagged and counted. The disability
gap, stated plainly. And nothing at a granularity that could identify a person —
which in this sector is the constraint that outranks every analytical preference.