Periods, and the aggregation operator nobody sets

Period types, and the identifier that carries them

Every value belongs to a period, and the period type is a property of the dataset the value was collected on. DHIS2’s period identifiers encode the type in the string, which is worth adopting even outside DHIS2:

202408      monthly
2024W32     weekly
2024Q3      quarterly
2024        yearly
2024April   financial year starting April

They sort correctly, they cannot be ambiguous between conventions, and the type is readable without a schema. The joining course made the case for ISO period keys; this is the same argument with the system’s own vocabulary.

import pandas as pd

vax = pd.read_csv("vaccination-coverage-2024.v1.csv", parse_dates=["period"])
vax["period_id"] = vax["period"].dt.strftime("%Y%m")
print(sorted(vax["period_id"].unique())[:4])

library(dplyr)
vax <- vax |> mutate(period_id = format(period, "%Y%m"))

Which period does a value belong to?

The question has one right answer and two wrong ones in daily use.

• Right: the period the activity happened in. A dose given on 28 August is August, however late the form arrives.
• Wrong: the period the form was submitted in. Common where data entry is batched, and it shifts activity forward by a month wherever reporting is late.
• Wrong: the period the value was keyed in. Same failure, one step further.

DHIS2 stores the value against the period the data entry form was opened for, so the system gets this right if the person opened the right form. The failure is human and it shows up as a characteristic pattern: a low month followed by a high one, which is the spike-and-dip the data quality course taught you to check for.

The three annual figures

Now the lesson. Take penta3 for the year and ask for annual coverage.

penta3 = vax[vax["antigen"] == "penta3"]
reported = penta3[penta3["report_submitted"] == True]

doses = reported["doses_administered"].sum()

summed_denominator = reported["target_population"].sum()
annual_target = penta3.groupby("facility_id")["target_population"].first().sum()

print(f"doses: {doses:,}")
print(f"a) doses / summed monthly target : {doses / summed_denominator:.1%}")
print(f"b) doses / annual target, all 38 : {doses / annual_target:.1%}")

penta3 <- vax |> filter(antigen == "penta3")
reported <- penta3 |> filter(report_submitted)

doses <- sum(reported$doses_administered)
summed <- sum(reported$target_population)
annual <- penta3 |> summarise(t = first(target_population), .by = facility_id) |>
  summarise(sum(t)) |> pull()

c(a = doses / summed, b = doses / annual)

Sentence	Figure
“Doses over target population, summed across the year”	6.5%
“Annual doses over the annual target population”	60.8%
“Annual doses over the annual target, among reporting facility-months”	77.5%

Three numbers, one file, one indicator, and each comes out of a sentence somebody would say in a meeting without blinking.

Why they differ

6.5% is a monthly figure wearing an annual label. target_population is an annual cohort — the surviving infants in the catchment for the year — and the extract repeats it in every month. Summing it across twelve months produces a denominator twelve times too large. The result is the average monthly coverage, and multiplying it by twelve gets you back to 78%.

60.8% counts silent facilities as having vaccinated nobody. The denominator is every facility’s full annual cohort, and the numerator is only the doses from facility-months that reported. 107 of 456 penta3 facility-months are missing, and this figure attributes zero doses to all of them.

77.5% is the defensible one, and it needs an explicit denominator adjustment:

months_reported = reported.groupby("facility_id").size()
targets = penta3.groupby("facility_id")["target_population"].first()
prorated = (targets * months_reported.reindex(targets.index).fillna(0) / 12).sum()

print(f"c) pro-rated denominator: {doses / prorated:.1%}")

months <- reported |> summarise(m = n(), .by = facility_id)
targets <- penta3 |> summarise(t = first(target_population), .by = facility_id)

prorated <- targets |>
  left_join(months, by = "facility_id") |>
  mutate(m = coalesce(m, 0)) |>
  summarise(sum(t * m / 12)) |> pull()

doses / prorated

Each facility contributes the share of its annual cohort matching the months it actually reported. It is coverage among reporting facility-months, and the label must say so.

The aggregation operator

Underneath all three figures is one configuration setting most analysts never see: how a data element aggregates over periods.

Operator	Meaning	Right for
Sum	Add across periods	Counts of events: doses, admissions, consultations
Average	Mean across periods	Stock levels, staffing, anything that is a state not an event
Last value	Take the most recent	Populations, targets, register sizes

doses_administered sums. target_population must not — it is a state, and it sums to twelve times itself. In a real instance the element would be configured as average or last value, and requesting annual coverage would then work. Here it is a repeated column and the correction is yours to make.

Ask for the aggregation operator with the metadata. It is one field per data element, it is invisible in the export, and it is the difference between 6.5% and 78%.

The same setting exists for aggregation across org units, where “sum” is almost always right and “average” is almost always wrong — a district’s doses are the sum of its facilities’, not their mean.

Deadlines, expiry and locking

Three settings that determine whether a value can still change.

• The deadline is when the dataset is due. It drives the timeliness figure the DQA course computes.
• Expiry days lock the form a fixed number of days after the period ends. After that, entry needs an unlock.
• A lock exception unlocks one dataset, one org unit, one period. Every one is a decision, and a system with hundreds of them has effectively no locking.

The consequence for an analyst: an extract of a recent period is provisional. Pull August in September and again in November and the numbers will differ, legitimately, because late entry is still arriving. That is not a data quality problem and it must not be reported as one — it is the reason the next lesson but one records an extraction date with every pull.

Completeness is a separate record

The last structural point, and it explains a number that otherwise looks inconsistent.

Completeness in DHIS2 is a registration, not a computation. When a user clicks “complete” on a data entry form, the system stores a completeness record for that dataset, org unit and period. The reporting rate is built from those records.

Which means a dataset can be complete with no values — somebody clicked complete on an empty form — and full of values but not complete — data entered, nobody clicked. Both happen constantly.

grid = (vax["facility_id"].nunique() * vax["period"].nunique()
        * vax["antigen"].nunique())
print(f"{grid} rows expected, {len(vax)} present, "
      f"{(vax['report_submitted'] == True).sum()} flagged reported")

c(expected = n_distinct(vax$facility_id) * n_distinct(vax$period) * n_distinct(vax$antigen),
  present = nrow(vax),
  reported = sum(vax$report_submitted))

2,736 rows present and 2,094 flagged as reported. The row exists either way, which is exactly the shape the cleaning course warned about — a missing report arriving as a zero with a flag beside it.

What comes next

The reporting flag is the raw material of the completeness figure, and the next lesson turns it into a denominator. The DQA course already showed why that matters; this one shows how the system computes it, and the two ways its answer differs from yours.