Reporting rate is a denominator, not a footnote — Routine Data and DHIS2 — Data Analysis

Four measures, one word

The Data Quality Assessment course established that reporting completeness is an indicator in its own right. This lesson is about what the system means by it, because DHIS2 exposes four related measures under names that get used interchangeably.

Measure	Numerator	Denominator
Reporting rate	Datasets marked complete	Datasets expected
Actual reports	Datasets marked complete	— (a count)
Expected reports	—	Org units assigned x periods
Reporting rate on time	Completed by the deadline	Datasets expected

The first and last differ only by timeliness, and a report that quotes “reporting rate” without saying which has left a reader unable to tell whether 82% means “submitted” or “submitted on time”.

import pandas as pd

vax = pd.read_csv("vaccination-coverage-2024.v1.csv", parse_dates=["period"])
vax["reported"] = vax["report_submitted"] == True

facility_months = vax.drop_duplicates(["facility_id", "period"])
print(f"expected: {len(facility_months)}")
print(f"actual:   {facility_months['reported'].sum()}")
print(f"rate:     {facility_months['reported'].mean():.1%}")

library(dplyr)

vax |>
  distinct(facility_id, period, .keep_all = TRUE) |>
  summarise(expected = n(), actual = sum(report_submitted),
            rate = mean(report_submitted))

456 facility-months expected, 349 submitted, 76.5%. Note the deduplication: completeness is registered per dataset per org unit per period, not per data element, so counting the 2,736 element-level rows would answer a different question and give the same percentage by coincidence.

Where the system’s answer differs from yours

Two differences, and both have caught people out.

Expected reports counts assignment, not existence. A facility that closed in March remains in the denominator for the rest of the year unless somebody un-assigns the dataset or closes the org unit. Reporting rates therefore drift downward as facilities close and nobody updates the assignment, and the drift looks like deteriorating performance.

Completeness is a click, not a calculation. A facility can enter every value and never mark the form complete; the system counts it as not reporting while the data sits there. The opposite also happens. So the reporting rate and the presence of data are two different questions:

has_data = (
    vax.groupby(["facility_id", "period"])["doses_administered"]
    .sum().gt(0).rename("has_data")
)
flags = facility_months.set_index(["facility_id", "period"])["reported"]

crosstab = pd.crosstab(flags, has_data.reindex(flags.index))
print(crosstab)

vax |>
  summarise(reported = first(report_submitted),
            has_data = sum(doses_administered) > 0,
            .by = c(facility_id, period)) |>
  count(reported, has_data)

On this extract the two agree exactly — every reported month has doses and every unreported month has none — which is the tidy case. Run the crosstab anyway. The off-diagonal cells are where a real instance keeps its most interesting problems, and a cell of “marked complete, no data” is a facility submitting empty forms to meet a target.

Reporting rate as the denominator of everything else

The operational point, and it is the reason this lesson sits in a DHIS2 course rather than only in the DQA one.

monthly = (
    vax[vax["antigen"] == "penta3"]
    .groupby(vax["period"].dt.strftime("%Y-%m"))
    .apply(lambda g: pd.Series({
        "reporting_rate": g["reported"].mean(),
        "coverage_reporting": (g.loc[g["reported"], "doses_administered"].sum()
                               / g.loc[g["reported"], "target_population"].sum()),
    }))
)
print((monthly * 100).round(1))

vax |>
  filter(antigen == "penta3") |>
  mutate(month = format(period, "%Y-%m")) |>
  summarise(
    reporting_rate = mean(report_submitted),
    coverage = sum(doses_administered[report_submitted]) /
               sum(target_population[report_submitted]),
    .by = month
  )

Month	Reporting rate	Coverage among reporters
July	82%	…
August	29%	…
September	45%	…
October	95%	…

Two columns, always adjacent. August’s coverage figure is computed on eleven of thirty-eight facilities, and whatever it says, it says it about those eleven.

Three rules that follow, and they are the deliverable of this lesson.

Below 90% reporting, a coverage figure is a lower bound, not an estimate. Label it “among reporting facilities” every time.
Do not compare two periods with materially different reporting rates without saying so. August against July is a comparison of eleven facilities with thirty-one.
A trend line needs the reporting rate on the same chart. Plotted alone, a coverage series through August looks like a programme collapse; plotted with completeness, the explanation is immediate.

The two ways to handle a silent facility

Neither is wrong. They answer different questions and they must be labelled.

Exclude it from both numerator and denominator. Coverage among facilities that reported. Honest, and it is what the code above does.

Impute its doses. Fill from the facility’s own recent months, and say so. This is what national estimates do — WUENIC and similar exercises impute rather than leave gaps, because the question is about children rather than about paperwork.

by_facility = (
    vax[(vax["antigen"] == "penta3") & vax["reported"]]
    .groupby("facility_id")["doses_administered"].median()
)
missing = (vax["antigen"] == "penta3") & ~vax["reported"]
imputed_total = (vax.loc[missing, "facility_id"].map(by_facility).sum()
                 + vax.loc[(vax["antigen"] == "penta3") & vax["reported"],
                           "doses_administered"].sum())

annual_target = (vax[vax["antigen"] == "penta3"]
                 .groupby("facility_id")["target_population"].first().sum())
print(f"imputed annual coverage: {imputed_total / annual_target:.1%}")

median_by_facility <- vax |>
  filter(antigen == "penta3", report_submitted) |>
  summarise(m = median(doses_administered), .by = facility_id)

Never impute silently. An imputed figure and a reported one in the same column with no flag is the defect the cleaning course named: an estimate that has stopped being distinguishable from a measurement.

Report the pair

summary = pd.DataFrame({
    "period": ["2024-08", "2024-09", "2024-10"],
    "facilities_reporting": [11, 17, 36],
    "facilities_expected": [38, 38, 38],
    "reporting_rate": ["29%", "45%", "95%"],
    "coverage_basis": ["among reporting facilities"] * 3,
})

tibble::tribble(
  ~period,   ~reporting, ~expected, ~basis,
  "2024-08",         11,        38, "among reporting facilities",
  "2024-09",         17,        38, "among reporting facilities",
  "2024-10",         36,        38, "among reporting facilities"
)

Five columns and no coverage figure travels without them. That is the whole argument, and it is cheaper than the meeting where somebody asks why August collapsed.

What comes next

Everything so far has run on a CSV somebody exported by hand. The next unit replaces that: the Web API, what to ask it for, and a script that makes the same request every month and records what it asked.