The 212 cases that are not a failure

The pathway has four gates

import pandas as pd

referrals = pd.read_csv("protection-referrals-2024.v1.csv")

gates = {
    "cases recorded": len(referrals),
    "consented to referral": referrals["consent_to_refer"].sum(),
    "referral made": (referrals["consent_to_refer"] &
                      referrals["referral_made"]).sum(),
    "referral accepted": (referrals["referral_made"] &
                          referrals["referral_accepted"]).sum(),
    "reached a service": (referrals["referral_accepted"] &
                          referrals["days_to_first_service"].notna()).sum(),
}
for name, count in gates.items():
    print(f"{name:24} {count:>5}")

library(dplyr)

referrals |> summarise(
  cases = n(),
  consented = sum(consent_to_refer),
  made = sum(consent_to_refer & referral_made),
  accepted = sum(referral_made & referral_accepted),
  reached = sum(referral_accepted & !is.na(days_to_first_service))
)

Gate	Cases	Share of the previous gate
Cases recorded	1,850	—
Consented to referral	1,638	88.5%
Referral made	1,142	69.7%
Referral accepted	757	66.3%
Reached a service	717	94.7%

This is the cascade from module 4’s first course, in a different sector — chained denominators, each gate measured against the one before it. The arithmetic is the same. The first gate is not.

Consent is not a performance measure

The 212 cases that did not consent could be counted three ways, and only one is defensible.

As failures. End-to-end completion becomes 717 / 1,850 = 38.8%. This treats a person’s decision as a programme shortfall.

As exclusions. Completion becomes 717 / 1,638 = 43.8% on cases that consented. This is the performance figure.

As a finding in their own right. 11.5% declined, and why is a service design question worth asking separately.

consenting = referrals["consent_to_refer"]
reached = referrals["referral_accepted"] & referrals["days_to_first_service"].notna()

print(f"on all cases:        {reached.sum() / len(referrals):.1%}")
print(f"on consenting cases: {reached.sum() / consenting.sum():.1%}")
print(f"declined:            {(~consenting).sum()} ({(~consenting).mean():.1%})")

referrals |> summarise(
  all_cases = mean(referral_accepted & !is.na(days_to_first_service)),
  consenting = sum(referral_accepted & !is.na(days_to_first_service)) / sum(consent_to_refer)
)

Counting a non-consenting case as a pathway failure does two wrong things at once. It misstates performance by five points, and it records a person’s autonomous decision as a defect in the system that offered them a choice.

Both matter and the second matters more. An indicator that treats declining as failure creates pressure on caseworkers to secure consent, which is the opposite of what informed consent means.

Report all three numbers

Referral pathway, 1,850 cases

  Consented to referral            88.5%   1,638
  Declined                         11.5%     212     reported separately
  Reached a service, of consenting 43.8%     717
  Reached a service, of all cases  38.8%             for reference only

Publish the consent rate as its own line. A falling consent rate is a signal about trust in the service, and it is invisible if consent is only ever used as a filter.

Where consent is not the gate

Two situations where this reasoning does not apply, both worth stating so the rule is not over-applied.

Child protection cases involving a young child operate under a best-interests determination rather than the child’s consent, and the consent field records the caregiver’s decision. The denominator logic is the same and the ethical basis is different.

Life-threatening emergencies proceed to a life-saving referral without waiting for consent to a data transfer. Those cases appear in the pathway and the consent column does not govern them.

by_category = referrals.groupby("case_category")["consent_to_refer"].agg(
    ["mean", "size"]
)
print((by_category * [100, 1]).round(1))

referrals |> summarise(consent = mean(consent_to_refer), n = n(),
                       .by = case_category)

Check whether the consent rate differs by category before applying one rule to all of them. Where it does, the denominator decision may need to differ too, and the report has to say which cases were treated which way.

The contradictions to resolve first

impossible = (referrals["days_to_first_service"].notna() &
              ~referrals["referral_accepted"])
no_referral = impossible & ~referrals["referral_made"]
no_consent = impossible & ~referrals["consent_to_refer"]

print(f"time to service but referral not accepted: {impossible.sum()}")
print(f"  of which no referral was made at all:    {no_referral.sum()}")
print(f"  of which there was no consent:           {no_consent.sum()}")

referrals |> filter(!is.na(days_to_first_service), !referral_accepted) |>
  count(referral_made, consent_to_refer)

Eleven cases record a time to first service on a referral that was never accepted. Six of them show no referral made at all, and one had no consent. These are logical contradictions and they have to be resolved before any completion rate is trusted, because each one is simultaneously a numerator and not a denominator.

The resolution is a judgement and it must be written down. Trusting the service date implies the pathway fields are unreliable; trusting the pathway fields implies the service date is a keying error. Say which you trusted and how many cases it moved.

What comes next

The pathway loses cases at every gate. The next lesson finds the gate where the largest share is lost, and the group of people for whom every gate is worse.