The two indicators nobody computes

Two indicators, and neither is in the report

Open almost any routine health report from this sector and you will find coverage, caseload, cure rates and stock-outs. You will very rarely find the two numbers that say how much of the district those figures describe.

They are cheap. Both come out of the extract you already have — one of them does — and both change how every other number in the report should be read.

Reporting completeness

The reporting rate is the share of expected reports that were actually submitted. The word doing the work is expected.

import pandas as pd

vax = pd.read_csv("vaccination-coverage-2024.v1.csv", parse_dates=["period"])
vax["reported"] = vax["report_submitted"] == True

print(f"overall reporting rate: {vax['reported'].mean():.1%}")

by_month = vax.groupby(vax["period"].dt.strftime("%Y-%m"))["reported"].mean()
print((by_month * 100).round(0))

library(dplyr)
library(readr)

vax <- read_csv("vaccination-coverage-2024.v1.csv")

mean(vax$report_submitted)

vax |>
  mutate(month = format(period, "%Y-%m")) |>
  summarise(reporting_rate = mean(report_submitted), .by = month) |>
  arrange(month)

Month	Reporting rate
January	92%
February	82%
March	74%
April–July	82–87%
August	29%
September	45%
October	95%
November–December	76–87%

August and September are not a wobble. Eleven of thirty-eight facilities reported in August and seventeen in September. Any August coverage figure is a statement about eleven facilities, and publishing it beside July’s without saying so compares two different districts.

Note also that the collapse and the recovery are district-wide and simultaneous. That is the signature of a system event — a supply of paper forms, a strike, a server, a supervisor who left — and not of facilities individually failing. A DQA that raises 27 separate facility findings for August has misread the pattern.

Break it down three ways, always

One number hides everything worth knowing. Break the reporting rate by time, by reporting unit, and by unit type.

by_facility = vax.groupby("facility_id")["reported"].mean().sort_values()
print(by_facility.head(6))
print(f"{(by_facility == 1).sum()} facilities reported every month")

by_type = vax.groupby("facility_type")["reported"].mean()
print((by_type * 100).round(1))

vax |> summarise(rate = mean(report_submitted), .by = facility_id) |> arrange(rate) |> head(6)

vax |> summarise(rate = mean(report_submitted), .by = facility_type)

Facility type	Reporting rate
Health centre	82.8%
District hospital	75.0%
Health post	71.7%

The worst six facilities all sit at 58% — seven months of twelve — and only two of thirty-eight reported every month. The gradient by type is the actionable finding: health posts report worst, which points at supervision distance, staffing or transport rather than at any individual facility.

The denominator trap

The one mistake that makes a reporting rate useless:

# WRONG: the share of reports that were submitted, among reports that were submitted
wrong = vax[vax["reported"]]["reported"].mean()          # always 1.0

# WRONG
vax |> filter(report_submitted) |> summarise(rate = mean(report_submitted))

Obvious written out. It is not obvious when the filter is six lines earlier in a pipeline, and it is exactly what happens when someone reads a DHIS2 export that only contains submitted reports. If the extract has no rows for non-reporting units, the reporting rate cannot be computed from it at all — you need the list of expected units from somewhere else.

EXPECTED_FACILITIES = pd.read_csv("facility-master-list.csv")["facility_id"]

expected = len(EXPECTED_FACILITIES) * vax["period"].nunique()
submitted = vax.loc[vax["reported"], ["facility_id", "period"]].drop_duplicates().shape[0]
print(f"{submitted} of {expected} expected facility-months")

expected  <- nrow(facility_master) * n_distinct(vax$period)
submitted <- vax |> filter(report_submitted) |> distinct(facility_id, period) |> nrow()

The master list is the authority, exactly as the administrative population frame was in the previous course. A facility that closed and one that stopped reporting look identical in the extract and are completely different findings, and only the master list tells them apart.

Timeliness needs a field you may not have

Timeliness is the share of reports submitted by the deadline, and the median lateness of the rest.

submissions["days_late"] = (
    submissions["submitted_on"] - submissions["deadline"]
).dt.days

print(f"on time: {(submissions['days_late'] <= 0).mean():.1%}")
print(f"median days late, of those late: "
      f"{submissions.loc[submissions['days_late'] > 0, 'days_late'].median():.0f}")

submissions <- submissions |>
  mutate(days_late = as.integer(submitted_on - deadline))

c(on_time = mean(submissions$days_late <= 0),
  median_late = median(submissions$days_late[submissions$days_late > 0]))

This extract cannot support that calculation, and saying so is itself a DQA finding. There is no submission timestamp in the file — only report_submitted, a flag with no date. DHIS2 records a lastUpdated per data value set and a completeness date per registration, so the field exists in the system and was not carried into the export.

Write that up as it stands:

Finding. Timeliness cannot be assessed. The monthly extract carries a submission flag but no submission date, although DHIS2 records one. Corrective action. Add the completeness date to the standard extract. Owner. HMIS focal point. By. Next quarterly extract.

A dimension you cannot measure is a finding about the reporting system, not a gap in your assessment. Report it in the same table as the rest.

Read every other figure through these two

The point of computing these first is what they license you to say afterwards.

• Coverage, at 76.5% reporting. A district total is a lower bound, not an estimate. Say “among reporting facilities” in the label, every time.
• A month-on-month trend. August’s fall is a reporting fall until proven otherwise. Plot the reporting rate on the same chart as the indicator, and the question answers itself.
• A facility ranking. Facilities reporting seven months of twelve cannot be ranked against facilities reporting twelve. Either restrict the ranking to a common set of months or rank on a per-month average and show the count.

monthly = (
    vax[vax["reported"]]
    .groupby(vax["period"].dt.strftime("%Y-%m"))
    .apply(lambda g: g["doses_administered"].sum() / g["target_population"].sum())
    .rename("coverage")
    .to_frame()
)
monthly["reporting_rate"] = by_month
print(monthly.round(3))

vax |>
  mutate(month = format(period, "%Y-%m")) |>
  summarise(
    coverage = sum(doses_administered[report_submitted]) /
               sum(target_population[report_submitted]),
    reporting_rate = mean(report_submitted),
    .by = month
  )

Two columns, always adjacent. A coverage figure without its reporting rate is an unlabelled number, and the label is the part that stops it being misread.

What comes next

Completeness and timeliness are about whether a report arrived. The next lesson is about whether the report was right — the recount against the source register, the verification factor it produces, and the tolerance band that turns it into a decision.