cassionData Analysis

Back to the lessonLesson 1 of 8A project that reopens

An analysis that reopens a year later

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 26

    What this lesson covers

    • The failure this lesson prevents
    • The three habits to drop
    • The project
    • here() rather than relative paths
    • renv: the package versions, recorded
    • Loading packages
    • Verifying the environment before trusting it
    • What to hand to a colleague
    • What comes next
    Speaker notes
    RStudio projects, here(), renv and the two habits — setwd() and a saved workspace — that make an R analysis run on exactly one machine on exactly one day.

  2. Slide 2 / 26

    The failure this lesson prevents

    • Someone asks you to explain last quarter's figure.
    Speaker notes
    Someone asks you to explain last quarter's figure. You open the analysis, run it, and get a different number — or it does not run at all, because the package that produced it has moved on two versions. That is not a bug in the analysis. It is the environment, and in R there are three specific habits that cause it.

  3. Slide 3 / 26

    The three habits to drop

    • setwd()

  4. Slide 4 / 26

    The three habits to drop — In R

    setwd("/Users/marie/Desktop/nutrition")

  5. Slide 5 / 26

    The three habits to drop

    • rm(list = ls()) at the top
    • Restart R instead — Ctrl/Cmd + Shift + F10 in RStudio
    • A saved workspace
    Speaker notes
    This runs on exactly one computer. It is the single most common reason a colleague's script fails on your machine, and the fix is not to edit the path — it is to stop having one. It looks like starting clean and is not. It removes the objects in your workspace and leaves everything else: attached packages, options, the working directory, anything loaded from a .Rprofile. A script that needs it is a script that is not reproducible, and running it in a fresh session is the only real test. Restart R instead — Ctrl/Cmd + Shift + F10 in RStudio. That is the clean start rm(list = ls()) pretends to be. RStudio offers to save .RData when you quit and to reload it when you start. Turn both off:

  6. Slide 6 / 26

    The three habits to drop — Example

    Tools -> Global Options -> General
  Restore .RData into workspace at startup    [ ]
  Save workspace to .RData on exit            Never
    Speaker notes
    A reloaded workspace means your session contains objects whose code you cannot see. The analysis appears to work because something from last week is still in memory, and it stops working the moment someone else runs it.

  7. Slide 7 / 26

    The project — Example

    muac-analysis/
  muac-analysis.Rproj
  data/
    raw/            the export exactly as it arrived, never edited
    interim/
  outputs/
    tables/
    figures/
  R/
    read_register.R
    indicator_table.R
  renv.lock
  README.md
  .gitignore
    Speaker notes
    An RStudio project is a directory with a .Rproj file in it. Opening the project sets the working directory to that directory, and that is the whole mechanism — but it is enough to make every path in your code relative to the project instead of to your home folder.

  8. Slide 8 / 26

    The project

    • data/raw is read-only — The export as it arrived is the only thing you cannot reproduce; everything else is…
    Speaker notes
    data/raw is read-only. The export as it arrived is the only thing you cannot reproduce; everything else is regenerated by running the code. So it is never edited, never sorted in Excel "just to look", and a correction lands beside the original as a new versioned filename rather than replacing it.

  9. Slide 9 / 26

    here() rather than relative paths — In R

    library(here)

muac <- readr::read_csv(here("data", "raw", "muac-screening-artibonite-2024.v1.csv"))
    Speaker notes
    A relative path like "data/raw/muac.csv" works from the project root and breaks in a notebook, in an R Markdown document knitted from a subdirectory, or when someone runs one line at a time from a different location.

  10. Slide 10 / 26

    here() rather than relative paths — In R

    here()
#> [1] "/home/marie/muac-analysis"
    Speaker notes
    here() finds the project root — the directory containing the .Rproj file, or a .here file, or a .git directory — and builds the path from there. The result is the same wherever the code is run from. Call here() once at the top of a script and use it everywhere. A path built by paste0() with a / in it will not work on Windows; here() handles the separator.

  11. Slide 11 / 26

    renv: the package versions, recorded — In R

    install.packages("renv")

renv::init()       # once per project
    Speaker notes
    here() fixes paths. It does nothing about the packages, and the packages are where the number changes.

  12. Slide 12 / 26

    renv: the package versions, recorded — In R

    renv::snapshot()   # after installing or upgrading anything
renv::restore()    # on another machine, or a year later
    Speaker notes
    renv::init() gives the project its own library and writes renv.lock, a record of every package and its exact version. From then on:

  13. Slide 13 / 26

    renv: the package versions, recorded

    • Commit renv.lock — It is the difference between "install the tidyverse" and "install the versions that produced this…
    • Installing where there is no internet
    Speaker notes
    Commit renv.lock. It is the difference between "install the tidyverse" and "install the versions that produced this table". The part most tutorials skip, and the part that matters on a deployment. On a connected machine, download the sources once:

  14. Slide 14 / 26

    renv: the package versions, recorded — In R

    renv::init()
renv::snapshot()

# Fill a local cache with everything the lockfile names.
renv::install()
renv::isolate()

  15. Slide 15 / 26

    renv: the package versions, recorded — In R

    # On the connected machine
dir.create("pkgs")
download.packages(c("dplyr", "readr", "haven"), destdir = "pkgs", type = "source")

# On the offline machine
install.packages(
  c("dplyr", "readr", "haven"),
  repos = NULL,
  type = "source",
  contriburl = paste0("file://", normalizePath("pkgs"))
)
    Speaker notes
    Then copy the project directory — including renv/library — to the field laptop. renv::restore() will use what is already there rather than reaching for a repository it cannot see. For a machine that will need packages you have not yet installed, the general mechanism is a local repository: Source packages need a compiler for anything with C or C++ in it, which is common. If the field machines are Windows, download the binaries instead (type = "win.binary") on a Windows machine of the same R version.

  16. Slide 16 / 26

    Loading packages — In R

    library(readr)
library(dplyr)

  17. Slide 17 / 26

    Loading packages

    • Do not use require() in a script — It returns FALSE and continues when the package is missing, so the script fails…
    • Do not call install.packages() from a script — A script that installs software as a side effect of being run is a…
    Speaker notes
    Two things not to do: Do not use require() in a script. It returns FALSE and continues when the package is missing, so the script fails later with a confusing error about an object that does not exist. library() stops there and tells you which package. Do not call install.packages() from a script. A script that installs software as a side effect of being run is a script nobody can safely run twice. Installation is renv::restore(), once, deliberately. Where two packages export the same name — dplyr::filter() and stats::filter(), dplyr::lag() and stats::lag() — say which you mean:

  18. Slide 18 / 26

    Loading packages — In R

    muac |> dplyr::filter(muac_mm < 125)
    Speaker notes
    The :: form is worth the characters in a script that will outlive your memory of what was attached.

  19. Slide 19 / 26

    Verifying the environment before trusting it — In R

    sessionInfo()

  20. Slide 20 / 26

    Verifying the environment before trusting it — In R

    stopifnot(getRversion() >= "4.2.0")
stopifnot(requireNamespace("dplyr", quietly = TRUE))
    Speaker notes
    For an analysis whose numbers matter, assert rather than inspect: |>, the native pipe used throughout this course, needs R 4.1 or later. If your team is on an older R, %>% from magrittr does the same job and the code in these lessons works unchanged with it.

  21. Slide 21 / 26

    What to hand to a colleague

    CommitDo not commit
    R/, .Rprojrenv/library/
    renv.lockdata/raw/ — see below
    README.mdoutputs/
    .gitignore.Rhistory, .RData

  22. Slide 22 / 26

    What to hand to a colleague — Example

    .Rproj.user/
.Rhistory
.RData
renv/library/
data/raw/
outputs/

  23. Slide 23 / 26

    What to hand to a colleague

    • Never commit raw beneficiary data — identified or pseudonymised
    Speaker notes
    Never commit raw beneficiary data, identified or pseudonymised. Git keeps every version of every file forever, and a repository that was internal on Monday can be shared on Friday. Credentials — a DHIS2 token, a KoboToolbox key — never appear in a script:

  24. Slide 24 / 26

    What to hand to a colleague — In R

    token <- Sys.getenv("DHIS2_TOKEN")
stopifnot(nzchar(token))
    Speaker notes
    Keep the value in a .Renviron file that .gitignore covers, and document the variable's name — not its value — in the README.

  25. Slide 25 / 26

    What comes next

    • The project reopens and the packages are pinned.
    Speaker notes
    The project reopens and the packages are pinned. The next lesson reads an export into it: readr for CSV, readxl for Excel, and the column specification that stops a facility code from becoming a number.

  26. Slide 26 / 26

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson