{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Attendance and the school feeding programme\n",
    "\n",
    "*School attendance, 2024 · R*\n",
    "\n",
    "Cassion · data-analysis.cassion.dev\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/thecassion/cassion-learning-platform/blob/main/apps/data-analysis/public/datasets/examples/school-attendance-2024/feeding-attendance.r.en.ipynb)"
   ],
   "id": "cell-000"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The question, and the answer that is too easy\n",
    "\n",
    "Do schools running a feeding programme have higher attendance? The raw means say\n",
    "yes, by about five points. The interesting part is how much of that five points\n",
    "survives being asked properly.\n",
    "\n",
    "Every dataset on this platform is synthetic. No real student is represented.\n",
    "\n",
    "## Setup"
   ],
   "id": "cell-001"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| message: false\n",
    "library(readr)\n",
    "library(dplyr)\n",
    "library(tidyr)\n",
    "library(ggplot2)\n",
    "\n",
    "BASE <- \"https://data-analysis.cassion.dev/datasets/files/\"\n",
    "\n",
    "attendance <- read_csv(paste0(BASE, \"school-attendance-2024.v1.csv\"),\n",
    "  col_types = cols(student_id = col_character(), present = col_character(),\n",
    "                   attendance_date = col_date()))\n",
    "\n",
    "roster <- read_csv(paste0(BASE, \"school-roster-2024.v1.csv\"),\n",
    "  col_types = cols(student_id = col_character(), school_id = col_character(),\n",
    "                   .default = col_guess()))\n",
    "\n",
    "dim(attendance); dim(roster)"
   ],
   "id": "cell-002"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clean the boolean and the duplicate registrations first\n",
    "\n",
    "One school used `Y` and `N`; two students appear on the roster twice after a\n",
    "transfer that was never de-registered."
   ],
   "id": "cell-003"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count(attendance, present)"
   ],
   "id": "cell-004"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attendance <- attendance |>\n",
    "  mutate(present_clean = case_when(\n",
    "    tolower(trimws(present)) %in% c(\"true\", \"y\", \"yes\")  ~ TRUE,\n",
    "    tolower(trimws(present)) %in% c(\"false\", \"n\", \"no\")  ~ FALSE,\n",
    "    TRUE ~ NA\n",
    "  ))\n",
    "\n",
    "roster_resolved <- roster |>\n",
    "  arrange(is.na(grade)) |>          # keep the registration that has a grade\n",
    "  distinct(student_id, .keep_all = TRUE)\n",
    "\n",
    "cat(\"roster rows:\", nrow(roster), \" unique students:\", nrow(roster_resolved), \"\\n\")\n",
    "\n",
    "daily <- attendance |>\n",
    "  inner_join(roster_resolved, by = \"student_id\") |>\n",
    "  filter(!is.na(present_clean))\n",
    "\n",
    "cat(\"student-days analysed:\", nrow(daily), \"\\n\")"
   ],
   "id": "cell-005"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exclude the strike so a closure does not read as absence\n",
    "\n",
    "Two schools have no attendance rows for fifteen school days in March. Because the\n",
    "rows are absent rather than false, they do not drag the mean down — but any\n",
    "analysis that reindexes to a full calendar would turn them into absences, and one\n",
    "of the two runs a feeding programme."
   ],
   "id": "cell-006"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "school_days <- daily |>\n",
    "  group_by(school_id) |>\n",
    "  summarise(days = n_distinct(attendance_date), .groups = \"drop\") |>\n",
    "  arrange(days)\n",
    "\n",
    "head(school_days, 4)"
   ],
   "id": "cell-007"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "closed <- school_days |> filter(days < max(days)) |> pull(school_id)\n",
    "\n",
    "roster_resolved |>\n",
    "  filter(school_id %in% closed) |>\n",
    "  distinct(school_id, feeding_programme)"
   ],
   "id": "cell-008"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "strike_window <- as.Date(c(\"2024-03-11\", \"2024-03-29\"))\n",
    "\n",
    "no_strike <- daily |>\n",
    "  filter(!(attendance_date >= strike_window[1] & attendance_date <= strike_window[2]))\n",
    "\n",
    "bind_rows(\n",
    "  daily     |> group_by(feeding_programme) |> summarise(scope = \"all days\",           rate = mean(present_clean), .groups = \"drop\"),\n",
    "  no_strike |> group_by(feeding_programme) |> summarise(scope = \"strike window dropped\", rate = mean(present_clean), .groups = \"drop\")\n",
    ") |>\n",
    "  mutate(rate = round(rate, 4)) |>\n",
    "  pivot_wider(names_from = feeding_programme, values_from = rate)"
   ],
   "id": "cell-009"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The gap barely moves, which is the reassuring outcome: the closure is invisible\n",
    "because the rows were never written. Run the check anyway — it is how you find\n",
    "out that the closure was handled correctly rather than assuming it.\n",
    "\n",
    "## The comparison that overstates itself"
   ],
   "id": "cell-010"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "daily |>\n",
    "  group_by(feeding_programme) |>\n",
    "  summarise(\n",
    "    student_days = n(),\n",
    "    attendance   = round(mean(present_clean), 4),\n",
    "    .groups = \"drop\"\n",
    "  )"
   ],
   "id": "cell-011"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Five points, on seventy thousand observations. It is tempting to test that\n",
    "directly, and the p-value would be spectacular — and meaningless.\n",
    "\n",
    "**The feeding programme is assigned to schools, not to students.** Seventy\n",
    "thousand student-days are not seventy thousand independent observations of the\n",
    "programme; they are twenty-four. Testing at the student-day level treats every\n",
    "child in a school as independent evidence about that school's programme, which is\n",
    "how a small effect acquires an impossible-looking p-value."
   ],
   "id": "cell-012"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "school_means <- daily |>\n",
    "  group_by(school_id, feeding_programme) |>\n",
    "  summarise(students = n_distinct(student_id), attendance = mean(present_clean), .groups = \"drop\")\n",
    "\n",
    "school_means |>\n",
    "  group_by(feeding_programme) |>\n",
    "  summarise(\n",
    "    schools   = n(),\n",
    "    mean_rate = round(mean(attendance), 4),\n",
    "    sd        = round(sd(attendance), 4),\n",
    "    .groups = \"drop\"\n",
    "  )"
   ],
   "id": "cell-013"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test at the unit the programme was assigned to"
   ],
   "id": "cell-014"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t.test(attendance ~ feeding_programme, data = school_means)"
   ],
   "id": "cell-015"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference survives — about five points — but the confidence interval runs\n",
    "from roughly 1.6 to 8.9 points. That is the honest precision of a comparison\n",
    "between nine schools and fifteen, and it is a very different claim from \"feeding\n",
    "raises attendance by 5.0 points\"."
   ],
   "id": "cell-016"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#| fig-width: 7\n",
    "#| fig-height: 4.5\n",
    "ggplot(school_means, aes(x = feeding_programme, y = 100 * attendance)) +\n",
    "  geom_boxplot(width = 0.45, outlier.shape = NA, colour = \"#5A6B66\") +\n",
    "  geom_jitter(width = 0.09, size = 2.2, colour = \"#2F5D50\", alpha = 0.8) +\n",
    "  labs(\n",
    "    x = \"School feeding programme\", y = \"Attendance (%)\",\n",
    "    title = \"Each point is a school, not a student\",\n",
    "    subtitle = \"24 schools is the sample size for this question\"\n",
    "  ) +\n",
    "  theme_minimal(base_size = 11) +\n",
    "  theme(panel.grid.minor = element_blank())"
   ],
   "id": "cell-017"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The overlap between the two groups is the finding. Several schools without a\n",
    "feeding programme out-attend several with one, so the programme is not the only\n",
    "thing driving attendance — and a school-level intervention decision needs to know\n",
    "that.\n",
    "\n",
    "## What this cannot establish\n",
    "\n",
    "Schools were not randomly assigned to the programme. If feeding went to schools\n",
    "that already had stronger management, better roads or more engaged parents, this\n",
    "comparison measures those things as well."
   ],
   "id": "cell-018"
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "school_means |>\n",
    "  arrange(desc(attendance)) |>\n",
    "  mutate(rank = row_number()) |>\n",
    "  select(rank, school_id, feeding_programme, students, attendance) |>\n",
    "  mutate(attendance = round(attendance, 3)) |>\n",
    "  head(10)"
   ],
   "id": "cell-019"
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nothing in this dataset lets you separate the programme from whatever selected\n",
    "schools into it. The defensible sentence is \"schools with a feeding programme\n",
    "attend about five points higher, 95% CI 1.6 to 8.9, in an unmatched comparison of\n",
    "24 schools\" — not \"feeding raises attendance by five points\".\n",
    "\n",
    "## What to report\n",
    "\n",
    "The effect with the interval from the school-level test, the number of schools on\n",
    "each side, the overlap between them, and the sentence saying assignment was not\n",
    "random. A programme evaluation that reports a student-day p-value has answered a\n",
    "question nobody asked."
   ],
   "id": "cell-020"
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "name": "R",
   "file_extension": ".r"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}