The EMIS nobody called a database

Everything this course taught was framed around a health information system. The concepts are not about health, and this lab proves it: a school attendance register has data elements, org units, periods and a completeness problem, and nobody has ever described it that way.

Doing so is not an academic exercise. The moment you can name the org unit level and the reporting rate, the figures stop being arguable — which is the whole point of the vocabulary.

The files

school-roster-2024.v1.csv — 1,202 rows covering 1,200 students in 24 schools. school-attendance-2024.v1.csv — 70,245 daily attendance marks over 60 school days. Both synthetic, and both carrying defects earlier courses have named.

Set up first

An RStudio project, both files read-only, an outputs/ directory, and a script that runs top to bottom from a clean session.

Part one: describe the system

Write the data model this register implies, in the vocabulary of the course. It is a short document and it is the first deliverable.

• The data elements. There is at least one and arguably three. Name them, say what each counts, and give each an aggregation operator over periods and over org units. At least one of them must not be summed.
• The org unit hierarchy. Student, school, and what above that? Say which level the analysis unit is and which level the reporting unit is — they are not the same, and that difference is the lab.
• The period type. The register is daily. What period type would this be collected on in a real system, and what does that do to the daily rows?
• The dataset. Which units are expected to report, on what schedule, and how would you know?

Part two: build the completeness picture

Compute, at the school-month level:

• expected reports, actual reports and the reporting rate
• the same three at school-day level, and say which of the two a report should quote
• the reporting rate by school, ordered worst first

Two schools have fifteen school days with no rows at all. In the vocabulary of this course, say precisely what that is — a completeness failure, a period that should not exist, or an org unit assignment problem — and defend the choice. The answer determines every figure after it.

Part three: the attendance rate, three ways

Produce three attendance rates for the district, exactly as lesson 3 produced three coverage figures:

• marks present over marks recorded
• marks present over enrolled students times all school days
• marks present over enrolled students times the days their own school was open

Report all three with their denominators, and say which one belongs in a report to the education office and why. Then say which of the three a naive query against a real EMIS would return, and what would have to be configured for it to return the right one.

Part four: what was counted

Write the provenance block from lesson 8 for the figure you chose in part three. Seven fields, every one filled from the data rather than from assumption. The Out of what and Completeness fields are the ones that will take the longest and they are the ones being marked.

What to hand in

An R script, sourced from a clean session, producing:

• the data model description as a printed table, one row per data element, with aggregation operators
• the completeness tables from part two
• the three attendance rates with denominators and n
• the provenance block as a text file in outputs/

Check your numbers

	Expected
School days in the term	60
Schools	24
Students enrolled	1,200
Attendance marks present	70,245 rows, of which some are not usable marks
Schools with fewer than 60 days of rows	2

If your enrolled-student count is 1,202, the roster’s duplicate rows are still there and every per-student denominator is wrong.

The questions to answer in prose

Three sentences each.

1. One of your data elements must not be summed over periods. Name it, say what its aggregation operator should be, and give the figure it produces if summed across the term.

2. The two schools with missing days are a completeness problem under one reading and a period problem under another. State which you chose, and what the other reading would do to those two schools’ attendance rates.

3. Your three attendance rates differ. Say which one an EMIS would return by default, whether that is the one the education office needs, and what the gap between them is in percentage points.

How to know you are done

Delete outputs/, restart R, one source(), and everything regenerates identically. Your provenance block has no field reading “unknown”, and every denominator in your output states the number of units behind it.

What this lab is not

It is not education statistics. Gross and net enrolment, survival to the final grade and what an attendance rate means for learning belong with the education content. This lab is the routine data model: naming what is counted, at what level and over what period, in a system that was never described as one — and proving that the vocabulary transfers.