# Feature definitions — School attendance and dropout early warning

Technical documentation for the project *School attendance and dropout early
warning*. It defines every feature the risk flag is built from, states the
threshold and where it comes from, and records the three data decisions that
change which students appear on the visit list.

- **Source datasets:** `school-attendance-2024.v1.csv` (70 245 attendance rows)
  and `school-roster-2024.v1.csv` (1 202 roster rows), 24 schools
- **Analysis:** `notebooks/dropout-early-warning.python.en.ipynb`
- **Decision informed:** which students receive a home visit before the end of
  term
- **Standards applied:** SDG 4.1

Every dataset on this platform is synthetic. No real student is represented.

---

## 1. What the model is, and is not

It is a **rule**, not a trained classifier: one threshold on one feature, chosen
so a head teacher can state why a student is on the list. There is no fitted
model, no training set and no accuracy figure, because there is no outcome label
— nobody in this dataset is recorded as having dropped out.

That is a limit, not an omission. A ranked list a head teacher can explain is
worth more here than a score they cannot, and any accuracy claim would be
fabricated.

---

## 2. Unit of analysis and the join

One row per student (1 200 after resolution), built from student-days.

| Step | Rule |
| --- | --- |
| Roster deduplication | Sort by `grade` (missing last), keep first per `student_id` |
| Join | `attendance` left-joined to the resolved roster, `validate="many_to_one"` |
| Analysis set | Student-days with a resolved present/absent value |

**Two students appear on the roster twice**, after an unregistered transfer left
both registrations live. `validate="many_to_one"` turns the resulting silent row
inflation into an error at the point it happens; without it, those two students'
attendance would be counted twice and both would be double-weighted in every
school rate.

The deduplication keeps the row with a recorded grade where one exists, because
grade is displayed to the head teacher.

Student-days analysed: **69 974** of 70 245 (271 rows carry no usable
present/absent value and are excluded).

---

## 3. The features

All features are computed **only on days the student's own school recorded**. See
§5 for why this is the load-bearing decision.

### 3.1 `days_recorded`

| | |
| --- | --- |
| **Definition** | Count of student-days with a resolved present/absent value |
| **Use** | Denominator of `attendance_rate`; a sanity check on thin records |

### 3.2 `attendance_rate`

| | |
| --- | --- |
| **Numerator** | Days present |
| **Denominator** | `days_recorded` |
| **Use** | Sorting the visit list within the flagged set — **not** the flag itself |

District-wide: **0.884**.

### 3.3 `trailing_absences`

| | |
| --- | --- |
| **Definition** | Length of the unbroken run of absences ending at the student's last recorded day |
| **Use** | Reported on the visit list |

### 3.4 `school_days_missed` — **the flag feature**

| | |
| --- | --- |
| **Definition** | Count of the student's recorded days falling after their last day present |
| **Denominator** | None — it is a count, not a rate |
| **Use** | The risk flag |

For a student never present, it is `days_recorded`.

### 3.5 `at_risk`

```
at_risk = school_days_missed > 15
```

15 **school days**, not calendar days — roughly three weeks of term. The figure
comes from the programme's own observation that attendance decays over about
three weeks before a student stops coming; that decay is the only reason early
warning is possible at all. It is a programme decision, and this document is
where it is recorded as such.

**Flagged: 72 of 1 200 students (6.0%).**

---

## 4. Why a recent run rather than a low total

The signal is a recent run of absence, not a low total. A student who missed a
fortnight in February and came back is not the same case as one who has missed
the last fortnight, and an attendance-rate threshold cannot tell them apart.

`attendance_rate` is used only to order the flagged set, so that a student who
attended at 95% and stopped three weeks ago sorts differently from one at 40% all
term. Sort order: `school_days_missed` descending, then `attendance_rate`
ascending.

---

## 5. The three data decisions that change the list

### 5.1 Attendance coding must be harmonised before anything is counted

One school recorded `Y`/`N`; the rest recorded `true`/`false`. A boolean cast
turns `Y`/`N` into missing values silently.

| Treatment | District attendance |
| --- | ---: |
| Naive boolean cast | 0.874 |
| Coding harmonised | 0.884 |

The one-point difference in the district rate understates the damage: **that
school then supplies most of the top of any risk ranking**, for a coding accident
rather than for dropout. The map is an explicit allow-list
(`true/TRUE/Y/y/yes` → present, `false/FALSE/N/n/no` → absent). Anything outside
it stays missing and is excluded — 271 rows.

### 5.2 A missing row is a closure, not an absence

**SCH07 and SCH18 were closed for 15 school days, 11 to 29 March 2024.** Those
student-days were never recorded — they are absent from the file entirely.

Nothing goes wrong until the features are built on a student-by-date matrix, the
natural shape for a run-length feature, because reindexing to the full district
calendar invents rows that were never recorded and the obvious fill is "absent".

| Treatment | Students flagged | Share from the two closed schools |
| --- | ---: | ---: |
| Closure filled as absent | 130 | 44.6% |
| Recorded days only | 85 | 15.3% |

Those two schools hold **9.8%** of the roster. Filling closures as absence makes
them 45% of the flags — the visit list becomes a list of children whose school
was shut.

**The fix is not an adjustment. It is not reindexing.** Every feature is computed
on the days the student's own school recorded, which is what §3 states.

### 5.3 The residual over-representation is reported, not adjusted away

After the fix, the two closed schools still account for **18.1%** of flags
against 9.8% of students. That gap is left in place and stated in the report: a
three-week closure is a plausible trigger for real disengagement, and adjusting
it away would remove exactly the students the programme most needs to visit.

School-level flag rates:

| School | Students | Flagged | Rate |
| --- | ---: | ---: | ---: |
| SCH07 | 61 | 7 | 11.5% |
| SCH24 | 54 | 6 | 11.1% |
| SCH18 | 56 | 6 | 10.7% |
| SCH15 | 47 | 5 | 10.6% |
| SCH09 | 49 | 4 | 8.2% |
| SCH14 | 48 | 4 | 8.3% |
| SCH01 | 55 | 4 | 7.3% |
| SCH22 | 56 | 4 | 7.1% |

A school whose flag rate sits well above the others is a conversation about the
school, not about fifteen separate students. Head teachers receive their own
school's list; the programme officer receives these counts.

---

## 6. Missing grade

Roughly 3% of roster rows have no grade. Those students are **still ranked** —
grade is not used in any feature. It appears on the list only so a head teacher
knows which class to ask about, and a missing grade must never remove a student
from a visit list.

---

## 7. What the list ranks

**Risk, not dropout.** A student on this list may have transferred, be ill, or be
helping at home during harvest. The output is a conversation to have before the
term ends, not a status to record — and a home visit that arrives treating the
student as a dropout will be the last one that family accepts.

---

## 8. Limitations

1. **No outcome label**, so no measurable sensitivity or specificity. The
   threshold is a programme judgement, and the honest framing is "students the
   programme wants to ask about", not "students who will drop out".
2. **Transfers are invisible.** A student who moved to another school looks
   identical to one who stopped attending. The two roster duplicates are evidence
   that transfer recording is imperfect.
3. **Absence has no reason code.** Illness, labour, cost and disengagement are
   indistinguishable, and they call for different responses.
4. **Term-end truncation.** A student who disengages in the final fortnight
   cannot accumulate 15 missed days before the term ends, and the rule will not
   flag them. Running it mid-term as well as at term end mitigates this.
5. **Closures are inferred from missing school-days**, not from a recorded
   calendar. A school that closed *and* failed to report would be
   indistinguishable from one that only closed.

---

## 9. Reproducing this analysis

```bash
pnpm examples:build
```

Then run `notebooks/dropout-early-warning.python.en.ipynb`. It needs pandas and
numpy and reads both CSVs over HTTPS.

Both datasets are versioned by filename and immutable; a correction ships as
`.v2.csv` with this document revised beside it.

---

## 10. Change log

| Date | Change |
| --- | --- |
| 2026-07-27 | First issue, against dataset v1. |