cassionData Analysis

Lab · Intermediate

Two figures that cannot drift

Build a small figures pipeline of your own, produce two charts for a WASH report from the committed survey, and prove the arrangement works by correcting the dataset and watching the chart and its caption move together.

PythonYour own machine180 min

A WASH report needs two figures: coverage by district, and the water quality distribution. You will build them through a pipeline rather than exporting them from a notebook, and the lab is finished when a change to the data moves both the chart and its caption without you touching either.

The file

wash-household-survey-2024.v1.csv — 2,403 households across three districts and eighteen communities. It carries the two defects the WASH course documented: one district written four ways, and eleven households whose total consumption was entered in a per-person column.

Set up first

project/
  data/            read-only, the committed CSV
  figures.py       one function per figure, one main()
  outputs/
    figures/       generated SVG and the CSV beside each
  report.md

Nothing is drawn by hand and nothing is exported from a notebook. python figures.py produces everything in outputs/, and deleting outputs/ costs nothing.

Part one: the pipeline skeleton

Write figures.py with a main() that creates the output directory and calls each figure function in turn. Each function must:

  • read from data/, never from a variable set elsewhere;
  • return or write both an SVG and a CSV of the values it drew;
  • take no arguments and depend on no global state.

Test it by deleting outputs/ and rerunning. If anything is missing, a figure depended on something the script did not produce.

Part two: coverage by district

Improved source, and basic service — improved and within thirty minutes — by district.

Normalise the district column first. The WASH course established that Nord-Ouest appears four ways; ungrouped, the worst district splits into fragments and none of them looks alarming. Do this in the figure function, not by editing the data.

Decide, and write the decision as a comment:

  • whether the two measures are grouped bars or two panels;
  • whether the districts are sorted by value or left in a fixed order;
  • what the axis maximum is.

Then apply lesson 7. The value axis starts at zero because these are bars. If your chart is unreadable at zero-based, the problem is the mark rather than the axis.

Part three: the water quality distribution

E. coli at the point of collection. The statistics course established the shape: mean 29.6, median 0, 53.6% of tested households at exactly zero.

This is the lab’s hard chart, because a histogram of a zero-inflated distribution is one enormous bar and eleven invisible ones.

Produce two versions and keep the better one, with the reason in a comment:

  • a bar chart of risk bands — none detected, 1–10, 11–100, over 100;
  • a histogram on a log scale, with the zeros handled explicitly.

Whichever you keep, the caption must state the share at zero. A summary of this variable that does not is the failure the statistics course opened with.

Part four: the captions

Write both captions as f-strings computed from the same objects the charts were drawn from. Each must carry the finding, the denominator, and one thing the figure does not support.

caption = (
    f"Improved water source against basic service, {n:,} households in "
    f"{districts} districts. Counting improved sources overstates coverage in "
    f"every district; the gap is households whose source is improved and more "
    f"than thirty minutes away."
)

No number in either caption may be typed. That is the property the last part tests.

Part five: prove it cannot drift

Add one line to the top of figures.py:

EXCLUDE_UNIT_ERRORS = True     # the 11 households with total litres in a per-person column

Run with it False, record every number in both captions, then set it to True and rerun.

Every affected number in both charts and both captions must change on its own. If a caption still says 2,403 households after the exclusion, that number was typed.

Check your numbers

Expected
Households 2,403
District values in the raw column 6
After normalising 3
Improved source — centre / nord-ouest / sud-est 88.0% / 72.9% / 78.2%
Basic service — same order 68.7% / 48.0% / 47.4%
E. coli tested 802 households
No detectable E. coli 430, or 53.6%
1–10 / 11–100 / over 100 173 / 139 / 60
Households above 80 litres per person 11

If your improved-source figures differ by district but your basic-service figures do not, the thirty-minute filter is being applied to the wrong column.

The questions to answer in prose

Three sentences each.

1. You chose between banded bars and a log histogram for E. coli. Name the choice, and say what a reader would take away from the version you rejected that is not true.

2. The district column had six values for three districts. Explain why normalising inside the figure function is better than fixing the CSV, in terms of what each choice means for the next person who runs the pipeline.

3. Setting EXCLUDE_UNIT_ERRORS to True changed some numbers and not others. List which, and say what that tells you about which figures the eleven rows were influencing.

What to hand in

  • figures.py, running from a clean checkout with python figures.py
  • two SVGs and two CSVs in outputs/figures/
  • report.md with both figures, their captions, and a methods note in the shape of lesson 8’s “report it whole”
  • a before-and-after table showing which numbers moved when the exclusion was applied
  • the three prose answers

How to know you are done

Delete outputs/, rerun, everything regenerates. Then open the CSV beside each figure and check that every number in the caption appears in it. A number in a caption that is not in the figure’s own CSV came from somewhere else, which is the failure this whole arrangement exists to prevent.

What this lab is not

It is not a lesson in matplotlib. Any plotting library does this, and the arrangement matters more than the tool — the platform’s own figures use no plotting library at all and emit SVG directly, which is a legitimate choice for two chart types and a bad one for twenty.

It is also not a WASH analysis. The coverage gap and the E. coli distribution are both findings the WASH and statistics courses established; here they are the material, and the deliverable is the pipeline that keeps the charts honest about them.