The panel that caught a wrong number in this course

What happened

The WASH project has carried an indicator definitions document since it was written. It states improved water coverage as 82.3% and basic service as 56.3%, with the numerator, the denominator and the standard for each.

The WASH course lesson, the committed figure and the first version of the project dashboard all said 79.7% and 54.7%.

# What the lesson had
IMPROVED = {
    "piped-into-dwelling", "piped-into-yard", "public-tap",
    "borehole", "protected-well", "protected-spring",
}

# What the JMP definition actually lists
improved <- c("piped-into-dwelling", "piped-into-yard", "public-tap",
              "borehole", "protected-well", "protected-spring",
              "tanker-truck")     # packaged or delivered water

The JMP improved list includes packaged or delivered water, which is tanker-truck in this dataset. The lesson omitted it — in a course that names the JMP ladder and SDG 6.1.1 explicitly — and understated improved coverage by 2.6 points and basic service by 1.6.

Nothing detected it for months. The lesson was internally consistent, the figure agreed with the lesson because it used the same set, the tests passed, and the number was plausible. It surfaced the day a second artefact computed the same indicator from the same file and disagreed.

Why the documentation was the one that was right

Because it stated the definition rather than implementing it.

### 2.1 Improved source

| Numerator   | Households whose main drinking water source is piped, borehole,
|             | protected well, protected spring, rainwater or packaged water
| Denominator | All surveyed households
| Value       | 0.823
| Valid at    | Household, community, district
| Standard    | WHO/UNICEF JMP service ladders

Writing “or packaged water” in prose made it checkable against the standard. A Python set of six strings does not read like anything; a numerator written as a sentence can be compared to the definition it claims to implement.

That is the whole argument for the panel. It is not documentation of the code — it is the specification the code is supposed to satisfy, written in a form a sector specialist can audit without reading Python.

What a definition entry contains

Six fields, and the last two are the ones usually missing.

Field	Example
Numerator	Households with an improved source and a round trip of 30 minutes or less
Denominator	All surveyed households
Value	0.563
Valid at	Household, community, district
Standard	JMP service ladders; SDG 6.1.1
Decision informed	Which communities receive rehabilitation

“Valid at” prevents the commonest misuse. A number computed at district level and quoted at community level is the same arithmetic answering a different question, and stating the level is what stops it.

“Decision informed” is what justifies the indicator existing. The WASH documentation writes “None. It is reported so the report can show what changed” against improved coverage — which is an unusually honest entry and exactly right: the figure is retained to quantify a gap, not to be used.

Where the panel lives

Three places, and they are not alternatives.

Where	What it holds
On the dashboard	One clause per tile — “improved and within 30 minutes”
In the technical documentation	The full entry, six fields, per indicator
In the code	A comment naming the standard and the reason

# The JMP improved list, which includes packaged or delivered water —
# `tanker-truck` here. Omitting it understates improved coverage by 2.6
# points and disagrees with the project's own indicator documentation.
IMPROVED = {...}

# The comment is what makes the next reader check rather than assume.

The tile clause is what a reader sees; the document is what a challenger reads; the comment is what the next analyst edits. All three, or the three drift.

The check the panel makes possible

Compute the indicator twice, from different code, and compare. That is what found the error here, and it is a check you can run deliberately rather than by accident.

from_pipeline = summary["basic_service"]
from_documentation = 0.563          # the value the definition panel states
assert abs(from_pipeline - from_documentation) < 0.001, (
    f"pipeline {from_pipeline:.3f} disagrees with the definition panel"
)

stopifnot(abs(basic_service - 0.563) < 0.001)

A stated value in the definition panel is a test fixture. Once the panel carries the number, any pipeline that computes it differently fails loudly — which is the workflows course’s rule arriving in a place nobody expects it.

And when they disagree, decide which is authoritative before changing either. Here the standard settled it: JMP lists delivered water, so the documentation was right and four other artefacts were corrected to match.

The argument this prevents every month

The indicator definition panel exists because of a specific recurring meeting.

“Coverage is 82%.” — “We reported 76% last quarter.” — “That was a different definition.” — “Which one is right?”

Both are right and they are different indicators. Without a panel that conversation takes twenty minutes every quarter and is never resolved; with one it takes as long as opening the document.

The commonest three disagreements in this sector, all of which a panel settles:

Source against service. Improved source, 82.3%. Basic service, 56.3%. Twenty-six points, and both are correct.

Numerator population against denominator population. Gross enrolment of 109.2% is a projected denominator, not more children than exist.

Screening against survey. A screening rate is not a prevalence estimate, and the nutrition dashboard here says so in its own “what this does not show” section.

Report it whole

Indicator definitions

  Full definitions in docs/indicator-definitions.md: numerator, denominator,
  the level it is valid at, the standard applied, and the decision it
  informs, for each of the nine indicators in this report.

  Each dashboard tile carries the definition compressed to one clause.

  The pipeline asserts its computed values against the values stated in the
  definitions document. A disagreement fails the build rather than reaching
  a report.

  Change log: 2026-07-29, improved source corrected from 79.7% to 82.3%.
  Delivered water (tanker truck) was omitted from the JMP improved list.
  The definitions document had it right; the analysis code did not.

The change log entry is the part worth copying. An indicator that changed definition needs a dated line saying what changed and which direction, or every comparison with an earlier report is silently wrong.

What comes next

The dashboard answers a question. The report is where the answer gets argued, and the next lesson is about writing one whose findings, limitations and recommendation can be read separately — because they will be.