Four things that change the answer when nothing changed

One: the clock

from datetime import date
report_date = date.today()          # different every day, by design

Anything that reads the current time makes every run differ. A generated-on footer, a filename with today’s date, a filter for “the last 30 days”, a random seed derived from the time.

The fix is to take the date from the data or from a parameter.

import pandas as pd

survey = pd.read_csv("data/raw/household-survey-2025.v1.csv")
as_of = pd.to_datetime(survey["interview_date"]).max()   # from the data

as_of <- max(as.Date(survey$interview_date))

This platform got this wrong and the failure is instructive. The course handouts and lesson decks originally took their generation date from the newest updated field across the entire content library. Deterministic — and still wrong, because publishing one new course restamped every existing handout and rewrote all ninety-six deck binaries.

The date is now scoped to the artefact. A handout’s date comes from its own course and lessons, a deck’s from its own lesson, a report’s from its own project. A deterministic wrong answer is still a wrong answer, and “the diff is enormous but correct” is how nobody reviews it.

Two: the seed

import numpy as np

rng = np.random.default_rng(20260729)      # stated, committed, reproducible
sample = rng.choice(households, size=200, replace=False)

set.seed(20260729)
sample(households, 200)

Anything random needs a seed and the seed belongs in the code. Bootstrap intervals, random sampling for verification, simulation, train/test splits, jittered scatter points.

Set it once, at the top, visibly. A seed buried three functions deep is a seed someone will move.

And say so in the output. The regression course’s bootstrap interval is reported as “400 resamples, seed 20260729” for exactly this reason — an interval nobody can reproduce is not an interval.

Every dataset on this platform is generated by a seeded script, which is what makes pnpm datasets:generate a verification step: it rewrites all twenty CSVs and git status reports nothing changed.

Three: the order files come back in

import glob
for path in glob.glob("data/raw/*.csv"):     # order is filesystem-dependent
    ...

glob and os.listdir return files in an order that differs between operating systems and sometimes between runs. If anything downstream depends on order — a concatenation, a first-wins deduplication, a row index — the result differs.

for path in sorted(glob.glob("data/raw/*.csv")):
    ...

for (path in sort(list.files("data/raw", full.names = TRUE))) { }

Sort it. Always. It costs six characters.

The same applies to dictionary and group order. groupby in pandas sorts by default and dplyr::group_by does not; a chart whose bar order came from an unsorted group is a chart whose order can change without the data changing.

Four: the locale

float("1,234")           # ValueError, or 1.234, depending on where you are

The decimal separator, the thousands separator, the date format and the sort order of accented characters are all locale-dependent, and this platform’s audience works across both conventions by definition.

survey = pd.read_csv(path, decimal=".", thousands=None)     # stated, not inferred
dates = pd.to_datetime(survey["date"], format="%Y-%m-%d")   # explicit

readr::read_csv(path, locale = locale(decimal_mark = ".", date_format = "%Y-%m-%d"))

State the format rather than letting the reader infer it. pd.to_datetime without a format is a function that guesses, and it guesses differently on 03/04/2025 depending on what else is in the column.

Sorting is the subtler one. sorted() on French commune names orders Étroit after Zone under one locale and before Fond under another, so a chart’s category order becomes machine-dependent.

The test that catches all four

python run.py && cp -r outputs outputs-first
python run.py && diff -r outputs outputs-first

Run it twice and diff. Anything that differs is one of the four, and the diff names the file.

Run it twice on different machines and you additionally catch the locale and the file ordering, which a single machine cannot.

Put it in CI, which is a clean machine every time, and the check runs whether or not anyone remembers.

The deterministic-output habit

For anything a build produces, byte-identical output on unchanged input is the goal, and it is achievable more often than people assume.

Source of churn	Fix
Timestamps in file metadata	SOURCE_DATE_EPOCH, or strip them
A generation date in a footer	Take it from the content
Compression with a timestamp	A zip whose entry times are pinned
Floating-point summation order	Sort before reducing where it matters
An embedded random ID	Derive it from a hash of the content

This platform pins SOURCE_DATE_EPOCH for both pdfTeX and pandoc, because a .pptx is a zip whose entry times and document properties would otherwise churn every deck on every rebuild.

The payoff is that a diff means something. When rebuilding produces no change, a change in the diff is a real change — and reviewing becomes possible.

Report it whole

Determinism

  All randomness is seeded: seed 20260729, set in src/config.py and reported
  with every bootstrap interval in this report.

  Dates are taken from the data (the latest interview date) rather than from
  the clock. No output contains a generation timestamp.

  File iteration is sorted. Group order is stated explicitly rather than
  inherited from the grouping library's default.

  CSV reading states the decimal mark and the date format rather than
  inferring them.

  Verified: running the pipeline twice produces byte-identical outputs, and
  CI runs it on a clean machine on every push.

The last line is the claim and the four above it are how it was achieved. A report that asserts determinism without saying which of the four it handled has probably handled the seed and none of the others.

What comes next

A reproducible pipeline that produces one report is worth having. The next lesson makes it produce twelve, from one template, without a copy anywhere.