A project someone else can open

The shape of a project

Almost every analysis in this sector has the same four kinds of file, and giving each a fixed home removes a category of question no one should have to ask.

muac-analysis/
  data/
    raw/            the export exactly as it arrived, never edited
    interim/        intermediate files, safe to delete
  outputs/
    tables/
    figures/
  scripts/
    read_register.py
    indicator_table.py
  notebooks/
    exploration.ipynb
  pyproject.toml
  uv.lock
  README.md
  .gitignore

The important line is the first one.

data/raw is read-only

The export as it arrived is the only thing in the project you cannot reproduce. Everything else — the cleaned table, the figures, the indicator tables — can be regenerated by running the code again. The raw file cannot: the server it came from will have moved on.

So it is never edited, never overwritten, never sorted in Excel “just to look at it”. If a correction arrives, it lands beside the original as a new file with a new name rather than replacing it.

That naming is worth being deliberate about:

data/raw/muac-screening-artibonite-2024.v1.csv
data/raw/muac-screening-artibonite-2024.v2.csv

not

data/raw/muac_final.csv
data/raw/muac_final_v2.csv
data/raw/muac_FINAL_corrected(1).csv

An analysis that ran on v1 must keep reproducing on v1. If the filename is overwritten, last quarter’s number can never be explained again — and explaining last quarter’s number is a thing you will be asked to do.

Paths that work on someone else’s machine

This is the single most common reason a colleague’s script fails on your laptop:

# Runs on exactly one computer.
muac = pd.read_csv("C:/Users/marie/Desktop/muac-screening-artibonite-2024.v1.csv")

The fix is to write paths relative to the project, and to let Python work out where the project is.

from pathlib import Path
import pandas as pd

# __file__ is this script; its parent is scripts/, whose parent is the project.
PROJECT = Path(__file__).resolve().parent.parent
RAW = PROJECT / "data" / "raw"
OUTPUTS = PROJECT / "outputs"

muac = pd.read_csv(RAW / "muac-screening-artibonite-2024.v1.csv")

Three things that buys you:

• It runs from anywhere. python scripts/indicator_table.py and python /home/marie/muac-analysis/scripts/indicator_table.py both work, because the paths are computed from the script’s own location rather than from the shell’s current directory.
• It works on Windows. Path joins with / in your source and produces \ on Windows. Never build a path by string concatenation.
• It is greppable. Every file the script touches is under RAW or OUTPUTS, so a reader can see the inputs and outputs at a glance.

In a notebook there is no __file__. Set the project root explicitly in the first cell instead, and keep the rest of the notebook relative to it:

from pathlib import Path

PROJECT = Path.cwd().parent      # notebook lives in notebooks/
assert (PROJECT / "data" / "raw").exists(), f"not the project root: {PROJECT}"

The assertion matters more than it looks. A notebook run from the wrong directory otherwise fails several cells later with a confusing error about a missing column.

Outputs are disposable, and that is the test

If deleting outputs/ and rerunning the scripts does not restore it exactly, something in the analysis is not reproducible — a manual step, an edit made in Excel, a cell run out of order.

rm -rf outputs/
uv run python scripts/indicator_table.py
git status                      # should show nothing unexpected

Make this a habit before handing anything over. It is the cheapest possible check and it catches the failure that is hardest to diagnose later.

Create output directories from the code rather than expecting them to exist:

OUTPUTS = PROJECT / "outputs" / "tables"
OUTPUTS.mkdir(parents=True, exist_ok=True)

indicator_table.to_csv(OUTPUTS / "gam_by_commune.csv", index=False)

parents=True creates intermediate directories; exist_ok=True means a second run does not fail. A colleague cloning the project gets the folders without being told to make them.

What goes in version control

Commit	Do not commit
scripts/, notebooks/	data/raw/ — see below
pyproject.toml, uv.lock	data/interim/
README.md	outputs/
.gitignore	.venv/, __pycache__/

.venv/
__pycache__/
*.pyc
data/raw/
data/interim/
outputs/
.ipynb_checkpoints/

Never commit raw beneficiary data, identified or pseudonymised. Git keeps every version of every file forever; a file deleted in a later commit is still in the history, and a repository that was internal on Monday can be shared on Friday. If the data is synthetic or already public, committing it is a convenience. If it is not, the raw directory stays out and the README says where the file comes from.

Credentials follow the same rule and are stricter: a DHIS2 token, a KoboToolbox API key or a database password never appears in a script, a notebook or a commit.

import os

# Read from the environment; the value never enters the repository.
token = os.environ["DHIS2_TOKEN"]

Keep the value in a .env file that .gitignore covers, and document the variable’s name — not its value — in the README.

Notebooks and scripts do different jobs

Both belong in a project and they are not interchangeable.

A notebook is for looking: exploring a new export, checking a distribution, producing a figure you will paste into a report. It runs out of order, holds state you cannot see, and diffs badly, which makes it a poor place for anything that must run identically next quarter.

A script is for producing: it runs top to bottom, takes arguments, and either succeeds or fails. When an exploration turns into a number someone will report, move it into a script.

The practical workflow is to explore in notebooks/, then move the settled part into scripts/ and import it back:

# scripts/read_register.py
from pathlib import Path
import pandas as pd

def read_register(path: Path) -> pd.DataFrame:
    return pd.read_csv(
        path,
        dtype={"child_id": "string", "commune": "string"},
        na_values={"muac_mm": ["-99"]},
    )

# In the notebook
import sys
sys.path.append(str(PROJECT / "scripts"))

from read_register import read_register

muac = read_register(RAW / "muac-screening-artibonite-2024.v1.csv")

Now the notebook and the script read the file the same way, and there is one place to fix when the sentinel changes.

A README that is actually read

Two commands and three sentences beat a page nobody finishes:

# MUAC screening indicator tables

Produces GAM and SAM rates by commune from the Artibonite screening register.

## Run

    uv sync
    uv run python scripts/indicator_table.py \
        --input data/raw/muac-screening-artibonite-2024.v1.csv \
        --output outputs/tables

## Data

data/raw is not committed. Download the register from the programme
share and place it there under its versioned filename.

What comes next

The project has a shape and the paths survive a change of machine. The next unit gets the data in: reading a CSV, an Excel workbook and a fixed-width export without losing a leading zero or misreading a date.