cassionData Analysis

Lesson 3 of 8

Unit · What travels with the number

As of when, from what, and how often

The three questions asked of every number on a dashboard, every month, by someone who was not in the analysis. Answering them on the page costs one line each and converts a recurring argument into a recurring glance.

PythonR135 minOECD DAC evaluation criteriaCore Humanitarian Standard (CHS)UNICEF indicator definitions

The three questions

They arrive in this order, from someone who trusts you and needs to defend the number to someone who does not.

“As of when?” A coverage figure with no date is a figure that will be quoted next March.

“From what?” Which export, which version, which extract of the database — and whether it is the same one the last report used.

“How often does it change?” Whether the number in front of them is this morning’s or last quarter’s, and whether checking again next week is worth doing.

Answer all three on the page. Each is one line, and the alternative is answering them by email eleven times.

What the answers look like

  Source: wash-household-survey-2024.v1.csv
  Data as of: 2024-11-30 (survey round 1)
  Refreshed: when a new survey round is published; this is a single round

Name the file, not the system. “From DHIS2” is where it came from; the export’s filename and version is what somebody can go and get.

Version by filename. .v1.csv is what makes “the same file the March report used” a checkable statement. A correction ships as .v2 and never overwrites, which is the rule the datasets on this platform follow and the reason a notebook pinned to v1 keeps reproducing.

Distinguish the data date from the build date. They are different and only one of them matters to a reader.

Means Put it on the page?
Data as of The last observation in the file Yes, prominently
Extract date When the export was pulled If they differ materially
Build date When the HTML was written No

The build date is the one everybody puts on and nobody needs. It changes on every rebuild, tells a reader nothing about the data, and — as the workflows course established — makes the output non-deterministic so a diff stops meaning anything.

None of the seven dashboards here carries a generation date, and that is why rebuilding them produces byte-identical files.

Refresh, stated as a cadence and an owner

  Refreshed monthly, on the 5th, from the screening database export.
  Owner: the M&E officer. If the export has not arrived by the 10th, the
  page still shows the previous month and says so.

A cadence without an owner is a wish. The useful line names who does it and what happens when it does not happen.

Say what a stale page looks like. The failure mode is a dashboard silently showing last month while everyone assumes it is current. A page that states its data date makes staleness visible without any machinery.

Completeness belongs next to the number

This is the provenance question this sector has that others do not.

submitted = reports[reports["report_submitted"]].shape[0]
print(f"reporting completeness: {submitted / len(reports):.1%}")
mean(reports$report_submitted)

A coverage figure computed from 76% of facilities is a different number from one computed from 99%, and the epidemiology course spent a lesson on a month where completeness fell to 28.9% and the case count fell with it.

So completeness goes on the page, beside any indicator whose denominator depends on who reported. The vaccination dashboard here carries it as a tile for exactly that reason.

And a month with no reports is drawn as a gap, never as a zero. Zero is a claim that nothing happened.

What the platform’s dashboards state

  Source: protection-referrals-2024.v1.csv
  How these numbers are produced: computed from the committed dataset file
  named above by scripts/export/build-project-dashboards.mjs. Nothing is
  typed by hand and the page contains no script, no external request and no
  generation date, so rebuilding it on unchanged data produces an identical
  file.

The last clause is the strongest provenance claim available, and it is checkable by anyone with the repository: rebuild, and git status is empty.

“Nothing is typed by hand” is the part that matters to a reviewer. A number in a caption that was typed can drift from the chart above it, and nothing detects it — which is the failure the visualisation course’s last lesson exists to prevent.

Where provenance is thin, say so

Three cases, and naming them is better than the alternative.

A number from another agency. “Cluster 4W, February submission” is provenance; “cluster data” is not. If you cannot version it, say when you received it and from whom.

A denominator from a projection. Population figures are usually projected from a census, and the projection’s base year is the provenance a reader needs. The education course’s gross enrolment ratio of 109.2% is a denominator story rather than an enrolment story.

A number somebody adjusted. If a facility’s obviously-wrong entry was corrected before analysis, that is a decision and it belongs in the technical documentation, with the page pointing at it.

Report it whole

Provenance panel

  Source            wash-household-survey-2024.v1.csv
                    2,403 households, 18 communities, 3 districts
  Data as of        2024-11-30, survey round 1
  Refresh           single round; no monthly cadence. A second round is
                    planned for 2025 and will make a trend meaningful.
  Completeness      water quality tested on 33.4% of households; every
                    quality figure states that denominator separately
  Produced by       scripts/export/build-project-dashboards.mjs, from the
                    committed file. No hand-entered numbers, no generation
                    date, byte-identical on rebuild.
  Definitions       docs/indicator-definitions.md

The last line is the handover to the next lesson. A provenance panel says where the number came from; a definition panel says what it is, and the two together are what stop the same argument recurring monthly.

What comes next

Two artefacts on this platform computed improved water coverage from the same file and got different answers. The next lesson is about the panel that would have caught it — and about the fact that it did.

Teach this lesson

The lesson as a slide deck, with the prose kept in the speaker notes rather than on the slide. Generated from this page, so it cannot fall out of step with it.

Start the slideshowRead the slides

The PDF needs no software and projects from any machine. The PowerPoint file is there to be edited — add your organisation's branding, cut a section for a shorter session, or merge two lessons into a workshop.