Back to the lesson·Lesson 3 of 8·Enrolled is not attending
88% attendance, 62% of students
The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.
What this lesson covers
- One register, two questions
- Average daily attendance
- The proportion regularly attending
- Which one a programme is judged on
- State the threshold, and where it came from
- The denominator decision nobody makes explicitly
- Report the pair
- What comes next
Speaker notes
Average daily attendance is 88.4%. The share of students attending at least 90% of their marked days is 62.1%. Both come from the same 70,245 rows, and the twenty-six points between them are the children a mean cannot see.One register, two questions — In Python
import pandas as pd attendance = pd.read_csv("school-attendance-2024.v1.csv") print(f"{len(attendance):,} rows, {attendance['student_id'].nunique():,} students") print(attendance["present"].value_counts(dropna=False))One register, two questions
- The
presentcolumn has five values, not two — SCH09 recordedYandNinstead oftrueandfalseon 873 of its…
Speaker notes
Thepresentcolumn has five values, not two. SCH09 recordedYandNinstead oftrueandfalseon 873 of its rows, and 271 days were never marked either way. A boolean cast turns those 873 real observations into missing values and silently thins one school out of the denominator.- The
One register, two questions — In Python
MARKS = {"true": True, "Y": True, "false": False, "N": False} marked = attendance[attendance["present"].isin(MARKS)].copy() marked["attended"] = marked["present"].map(MARKS) print(f"usable marks: {len(marked):,} of {len(attendance):,}")One register, two questions — In R
attendance |> mutate(attended = case_when(present %in% c("true", "Y") ~ TRUE, present %in% c("false", "N") ~ FALSE)) |> filter(!is.na(attended))Speaker notes
Now the same file answers two different questions.Average daily attendance — In Python
ada = marked["attended"].mean() print(f"average daily attendance: {ada:.1%}")Average daily attendance
- 88.4% — Every mark counts once, so a student present 60 days out of 60 and one present 30 out of 60 contribute in…
Speaker notes
88.4%. Every mark counts once, so a student present 60 days out of 60 and one present 30 out of 60 contribute in proportion to how often they were marked. This is the number a ministry reports and a system-level indicator should be. It answers how full is the classroom on an average day, which is the right question for staffing, feeding and textbook planning.The proportion regularly attending — In Python
by_student = marked.groupby("student_id")["attended"].agg(["mean", "size"]) eligible = by_student[by_student["size"] >= 20] for threshold in (0.80, 0.85, 0.90): share = (eligible["mean"] >= threshold).mean() print(f"attending at least {threshold:.0%} of marked days: {share:.1%}")The proportion regularly attending — In R
marked |> summarise(rate = mean(attended), days = n(), .by = student_id) |> filter(days >= 20) |> summarise(across(everything(), ~ mean(rate >= 0.9)))The proportion regularly attending
Threshold Students meeting it At least 80% of days 87.3% At least 85% 78.9% At least 90% 62.1% The proportion regularly attending
- 62.1% at the 90% threshold, against an average daily attendance of 88.4% — The two numbers are twenty-six points apart…
- A mean over marks describes the system; a proportion over students describes children — A school where every child…
Speaker notes
62.1% at the 90% threshold, against an average daily attendance of 88.4%. The two numbers are twenty-six points apart and neither is wrong. A mean over marks describes the system; a proportion over students describes children. A school where every child misses one day in eight and a school where seven children in eight attend perfectly while one never comes have the same average daily attendance and completely different problems.Which one a programme is judged on
- Report both, and lead with the one that matches the decision
Speaker notes
Report both, and lead with the one that matches the decision.Which one a programme is judged on
The decision The number How many meals, desks, textbooks Average daily attendance Which children need a follow-up visit Proportion below the threshold Whether an intervention worked Both, because they can move in opposite directions Which one a programme is judged on — In Python
struggling = eligible[eligible["mean"] < 0.75] print(f"students below 75%: {len(struggling)} " f"({len(struggling) / len(eligible):.1%})") print(f"their marks as a share of all marks: " f"{struggling['size'].sum() / eligible['size'].sum():.1%}")Speaker notes
That last row is the one to internalise. An intervention that brings the worst attenders from 40% to 60% moves average daily attendance barely at all and moves the proportion above 90% not at all — and would be recorded as a failure by either number alone.Which one a programme is judged on — In R
# The students furthest behind are a small share of the rows and the whole # of the problem.State the threshold, and where it came from — In Python
sensitivity = {f"{t:.0%}": f"{(eligible['mean'] >= t).mean():.1%}" for t in (0.75, 0.80, 0.85, 0.90, 0.95)} print(sensitivity)Speaker notes
The 90% threshold is a convention, not a standard, and it does most of the work in that 62.1%. Move it to 85% and the figure becomes 78.9%.State the threshold, and where it came from — In R
# Print the curve, not the point.Speaker notes
At 95% it is 39.8% and at 75% it is 90.5%. Publish the threshold beside the number, every time. Two reports quoting "the proportion of students regularly attending" at different thresholds produce incomparable figures that both look official — the same failure as the two Food Consumption Score threshold sets, in a different sector.The denominator decision nobody makes explicitly
- Without it — a student who enrolled in the last week of term and attended three days out of three appears as a 100%…
- With it — you have excluded exactly the late-arriving and early-leaving students, who are the ones an attendance…
Speaker notes
by_studentabove was filtered to students with at least twenty marked days. That is a choice and it has to be declared. Without it, a student who enrolled in the last week of term and attended three days out of three appears as a 100% attender, and a student marked twice appears in the tail. With it, you have excluded exactly the late-arriving and early-leaving students, who are the ones an attendance programme most wants to see.The denominator decision nobody makes explicitly — In Python
print(f"students with any marks: {len(by_student)}") print(f"students with 20 or more: {len(eligible)}")The denominator decision nobody makes explicitly — In R
# Two denominators, both defensible, and the report says which.Speaker notes
Here the two are nearly identical, so the choice does not move the answer. In a register covering a full year it would move it a great deal, and the habit of declaring it is what makes the figure portable.Report the pair — Example
Attendance, February to April 2024 Average daily attendance 88.4% 69,974 usable marks Students attending 90%+ of days 62.1% 1,200 students with 20+ marks Students attending 85%+ 78.9% Students below 75% 9.5% 114 students, the follow-up list SCH09 recorded 873 marks as Y/N rather than true/false; recoded, not dropped. 271 days were never marked and are excluded from both figures.What comes next
- Both numbers in this lesson divide by days that were marked.
Speaker notes
Both numbers in this lesson divide by days that were marked. The next lesson is about the days that are not in the file at all — fifteen of them, at two schools, which turn a strike into a dropout emergency if you let them.