Git does not forget, which is the point and the danger

The commit that cannot be taken back

git add data/raw/beneficiaries.csv
git commit -m "add survey data"
# ... realised three days later ...
git rm data/raw/beneficiaries.csv
git commit -m "remove data"

The file is still in the repository. It is in the history, it is in every clone, it is on the remote, and it is in the local copy of everyone who pulled. The second commit removed it from the current state and from nothing else.

On a repository holding protection or health data, that is a disclosure. Not a mistake to fix later — a disclosure, with the same obligations as any other.

Which is why the rule is preventive rather than corrective. You cannot un-commit personal data; you can only stop it entering.

The .gitignore you write before the first commit

# Raw data. Nothing under here is ever committed.
data/raw/*
!data/raw/.gitkeep
!data/raw/CHECKSUMS

# Derived data, regenerated by the pipeline
outputs/

# Credentials, in every form they arrive in
.env
*.pem
*_secret*
config.local.*

# Editor and language noise
.Rhistory
.RData
__pycache__/
*.pyc
.DS_Store
.ipynb_checkpoints/

Write it first, commit it first. A .gitignore added after the first commit is a .gitignore that arrived too late for the thing it was meant to catch.

The negations matter. data/raw/* excludes everything, then !data/raw/CHECKSUMS lets back exactly the file that proves which export was used — provenance without content.

What “personal data” means here, precisely

It is broader than a name column, and this sector’s datasets make the point.

Looks anonymous	Is not, because
A case ID	It joins to a case management system that has the name
GPS coordinates of a household	It is an address
Date of birth plus commune plus sex	Three fields identify most people in a small commune
A photograph of a form	Every field on it
A free-text “notes” column	It contains whatever the caseworker typed

The protection course established the arithmetic: a four-way disaggregation of an anonymous dataset produced nineteen cells of one, and a cell of one is an identification.

So the test is not “does it have a name column”. It is whether any combination of what the file holds could pick one person out — and in a small commune it usually can.

The check that runs before the commit does

# .git/hooks/pre-commit — or the pre-commit framework, or a CI step
if git diff --cached --name-only | grep -qE '^data/raw/'; then
  echo "Refusing to commit anything under data/raw/" >&2
  exit 1
fi

# The same check as an R function, run by whatever CI you have.

A hook that refuses is worth more than a policy that reminds. The failure mode is someone in a hurry at 6pm, and a reminder does not survive that.

Add a scan for the shapes secrets take, because a token pasted into a script is the other common case. gitleaks and detect-secrets both do this in a CI step.

What to do if it has already happened

The order matters and the first step is not technical.

One: report it. Your organisation has a data-incident procedure. This is one, and whether the repository was public decides how urgent rather than whether.

Two: rotate anything that was a credential. A committed token is compromised permanently; deleting it changes nothing.

Three: rewrite the history, with git filter-repo or BFG, and force-push. Every existing clone must be deleted and re-cloned — a clone nobody re-cloned still has the file.

Four: assume it was fetched. On a public repository, assume the file was retrieved and mirrored, and treat the disclosure as complete.

git filter-repo --path data/raw/beneficiaries.csv --invert-paths

Step three is the only one that looks like a fix and it is the least important.

Commits on analysis work

The habits differ from software work in two ways worth naming.

Commit the code and the output together when the output is committed. A figure and the script that made it belong in one commit, so a reviewer can see whether the chart matches the change.

Write the why in the message. fix commune names is what the diff already shows. “Normalise four spellings of Nord-Ouest; ungrouped, the worst district splits into four fragments and none of them looks alarming” is the thing a successor needs and cannot recover.

git commit -m "Group four spellings of Nord-Ouest before computing coverage

Ungrouped, the worst district splits into fragments of 727, 33, 22 and 20
households and none of them looks alarming. The 20-household fragment shows
0% open defecation, which a district table would report as a solved problem."

Branch per analysis question, not per day. A branch that answers “does coverage differ by district” can be reviewed and merged; a branch called work cannot.

What a repository of programme analysis should contain

project/
  .gitignore          committed first
  README.md           what it produces, how to run it, what to know
  uv.lock             the environment, pinned
  src/                every transformation
  tests/              the checks that fail loudly
  data/
    raw/              .gitkeep and CHECKSUMS only
    reference/        committed: thresholds, admin codes, lookup tables
  outputs/            ignored

Reference tables are committed and raw data is not, and the line between them is whether the file describes people. WHO growth standards, IPC thresholds and admin boundary codes are reference; a household list is not.

Report it whole

Data handling in this analysis

  No raw data is committed. data/raw/ is excluded by .gitignore, with only a
  CHECKSUMS file tracked so the exports used can be identified.

  A pre-commit hook refuses any staged file under data/raw/ and a CI step
  scans for credential patterns.

  Reference tables (WHO growth standards, IPC thresholds, admin codes) are
  committed; they describe no individual.

  The repository is private and access is limited to the M&E team. Raw
  exports are held in [the organisation's storage], not here.

The last line is the one that gets forgotten and it is the one an audit asks for. Saying where the data is matters as much as saying where it is not.

What comes next

Code in version control still does not reproduce a number if the environment differs. The next lesson pins it, and finds the four things that make a rerun differ when nothing in the repository changed at all.