Lab · Intermediate
The finding the coding hid
One area recorded disability as Yes and No instead of true and false, splitting the equity finding into four categories and leaving a cell of three cases at 0%. Clean the caseload, defend the denominator, and write a log you could attach to a protection report.
This caseload exists to answer one question: does the referral pathway work equally well for people who report a disability? The answer is in the file. A coding inconsistency in one area, forty-one cases wide, is standing in front of it.
Run the disaggregation without cleaning and you get four categories rather than two, one of them three cases with a completion rate of 0%. Someone will put that cell in a slide.
This lab runs locally rather than in a hosted notebook, and that is deliberate. Protection data does not go into a hosted session, and the habit is worth having even when the file is synthetic.
The file
protection-referrals-2024.v1.csv — 1,850 synthetic protection cases tracked
from intake to first service across three areas. It carries no names, no contact
details, no free text, no incident date, no location below admin2 and no exact
age, because none of those are needed to measure whether a pathway works.
Read the dataset’s data quality notes before you start. What it omits is part of the material.
Set up first
A project folder with the raw file read-only, an outputs/ directory, and a
script that runs top to bottom. You will run it twice — once now, once at the end
from a clean interpreter — and the second run is the one that counts.
The task
Produce a defensible referral completion rate, disaggregated by disability status, area and service, with a cleaning log attached.
Six things stand in the way.
Four values in a boolean column. disability_reported holds true, false,
Yes and No. All forty-one Yes/No cases are from one admin2 area. Normalise
before you group, and check the count of levels afterwards rather than assuming
the recode worked.
The consent gate. A case that did not consent to referral is not a pathway failure — it is a person’s decision. The completion denominator is consenting cases, not all cases. Two hundred and twelve cases did not consent, and putting them in the denominator understates the pathway by about five points while misrepresenting what those people chose.
Eleven contradictions. Eleven cases record a time to first service although the referral was never accepted. Six of those also record no referral made at all, and one had no consent. Decide what each state means, and note that “resolve them all the same way” is a decision you have to justify.
Two denominators, not one. Forty accepted referrals have no time to first service recorded. Completion and timeliness therefore have different denominators. Report each against its own and say which is which in the column name.
Sixty-two blank age bands. Age is the field intake staff miss most often and the one most disaggregations depend on. Show whether the blanks cluster by area, month or case category before you decide what to do with them.
Small cells. Some combinations of area, service and disability status contain fewer than five cases. Decide your suppression rule before you look at the numbers, not after.
What to hand in
A Python script, run from a clean interpreter, producing:
- a completion table by disability status, on the consent-gated denominator, with the number of consenting cases shown beside every rate
- the same table by admin2 and by service requested, with cells below your suppression threshold suppressed rather than printed
- a timeliness table on its own denominator, with the share of accepted referrals missing a service date stated
- a cleaning log CSV with one row per rule — rule, rows, action, effect on the completion rate, decided by, date
- the arrival profile from lesson 1, saved as JSON
Check your numbers
The dataset notes state what a correct analysis finds.
| Expected | |
|---|---|
| Cases that consented | about 88.5% |
| Completion, consenting cases | about 46% |
| Completion, disability reported | about 31% |
| Completion, no disability reported | about 48% |
| Completion by service | 23% (livelihood) to 62% (health) |
If your disability disaggregation has more than two rows, the recode did not run before the grouping. If overall completion is nearer 41%, you have the whole caseload in the denominator instead of the consenting cases.
The questions to answer in prose
Three sentences each.
1. Before cleaning, the Yes category contains three cases and completes at
0%. Say why that cell is uninterpretable rather than alarming, and what a reader
would have concluded from a slide showing it.
2. All forty-one Yes/No records come from one admin2 area. Explain what that tells you about where the defect came from, and what you would do about it that is not a code change.
3. You suppressed some cells and reported others. State your rule, and explain why a suppression rule chosen after seeing the numbers is not a suppression rule.
How to know you are done
Delete outputs/, restart the interpreter, run the script once, and every table
and the log are regenerated identically. Your cleaning log contains no case
identifier in a sentence that also names a case category and an area — check that
by reading it, because nothing in your code will.
What this lab is not
It is not referral pathway analysis. Why completion is the right indicator, what a pathway node is and how the disability gap should be interpreted belong with the protection content. This lab is about the cleaning: a recode that must run before the grouping, a denominator that carries an ethical claim, contradictions that need a decision rather than a filter, and a log you could attach to a report without creating a protection risk.