The dplyr verbs, used deliberately

Five verbs and a pipe

library(dplyr)

muac |>
  filter(!is.na(muac_mm)) |>
  mutate(gam = muac_mm < 125) |>
  group_by(commune) |>
  summarise(assessed = n(), cases = sum(gam)) |>
  arrange(desc(cases / assessed))

Each verb takes a data frame and returns one. The pipe passes the result along. That is the whole grammar, and the rest of this lesson is about what each verb does when the data is not clean — which is always.

|> is R’s native pipe, available since 4.1. %>% from magrittr predates it and behaves near-identically for this kind of code; if your team’s scripts use it, everything here works unchanged.

filter() drops missing values, silently

nrow(muac)
#> [1] 4218

nrow(filter(muac, muac_mm < 125))
#> [1] 319

sum(is.na(muac$muac_mm))
#> [1] 72

A comparison against NA yields NA, and filter() keeps only rows that are TRUE. So those 72 unmeasured children are gone, and nothing said so.

This is the same behaviour as pandas, and it is worth stating in both languages because it is the fastest way to change a denominator by accident. Be explicit about which you meant:

# Children measured below 125 mm.
filter(muac, muac_mm < 125)

# Children not known to be at or above 125 mm — the unmeasured included.
filter(muac, is.na(muac_mm) | muac_mm < 125)
#> 391 rows

Seventy-two rows separate those two questions. Which one you want depends on whether an unmeasured child is a non-case or an unknown, and that is a decision for a cleaning log, not a default.

select() and the helpers

muac |> select(child_id, commune, muac_mm)
muac |> select(-oedema)
muac |> select(starts_with("muac"), ends_with("_date"))
muac |> select(where(is.numeric))

select() renames as it selects, which is the tidy way to fix an export’s column names once at the top:

epi |> select(facility = facility_id, month = period, doses = doses_administered)

Two habits worth forming. Select before you join, so the result does not carry forty columns you did not want. And never select by position — select(3:5) breaks the day the export gains a column, and it breaks silently because the code still runs.

mutate() and the two conditional functions

muac <- muac |>
  mutate(
    gam = muac_mm < 125,
    sam = muac_mm < 115,
    band = case_when(
      muac_mm < 115 ~ "severe",
      muac_mm < 125 ~ "moderate",
      .default = "normal"
    )
  )

case_when() evaluates in order and the first match wins, so < 115 must come first. Its .default catches everything else including NA, which is almost never what you want:

case_when(c(110, 130, NA) < 125 ~ "case", .default = "not")
#> [1] "case" "not"  "not"

The missing measurement became "not". Handle it explicitly:

band = case_when(
  is.na(muac_mm) ~ NA_character_,
  muac_mm < 115  ~ "severe",
  muac_mm < 125  ~ "moderate",
  .default = "normal"
)

Putting the is.na() branch first is the habit. This is the same trap as np.select’s default in the Python course, and it is worth recognising in both places.

if_else() rather than ifelse()

ifelse(c(TRUE, FALSE), 1L, "x")
#> [1] "1" "x"

Base R’s ifelse() silently coerced an integer and a string into a character vector. A column you believed was numeric is now text, and you find out three steps later when a sum fails.

if_else(c(TRUE, FALSE), 1L, "x")
#> Error: Can't combine `true` <integer> and `false` <character>.

dplyr’s if_else() refuses. Prefer it everywhere, for the reason this course keeps returning to: an error where the mistake is made beats a wrong answer three steps later.

if_else() also takes a missing = argument, which is how you say what an NA condition should produce rather than letting it propagate.

across(): one rule, many columns

muac |>
  summarise(across(c(muac_mm, age_months), ~ mean(.x, na.rm = TRUE)))
#> # A tibble: 1 × 2
#>   muac_mm age_months
#> 1    140.       30.9

across() is what stops a cleaning script being fifteen near-identical lines.

survey |> mutate(across(where(is.character), stringr::str_squish))

survey |> mutate(across(starts_with("fcs_"), ~ if_else(.x > 7, NA_real_, .x)))

That second line is the food-security rule from the Python for Programme Data course, in one statement: a consumption value above seven days is impossible and must become missing rather than being clipped, across all eight food groups at once.

The ~ .x form is a shorthand for a function of one argument. \(x) ... is the newer base R spelling and works identically:

survey |> mutate(across(starts_with("fcs_"), \(x) if_else(x > 7, NA_real_, x)))

arrange()

muac |> arrange(commune, desc(screening_date))

NA sorts last regardless of direction, which is usually what you want and occasionally hides a problem — a column that is entirely missing looks sorted.

Sorting arranges rows. It says nothing about whether the differences between adjacent rows are real, a point the Nutrition programme dashboard project makes at length where nine of twelve communes have overlapping confidence intervals.

distinct() and counting

n_distinct(muac$commune)
#> [1] 12

muac |> distinct(child_id, .keep_all = TRUE)

distinct() without .keep_all = TRUE returns only the named columns, which surprises people expecting deduplicated rows. With it, the first row of each group is kept — so sort first if which one survives matters.

muac |> count(commune, sort = TRUE)

count() is group_by() |> summarise(n = n()) |> ungroup(), and it is the right tool whenever that is all you want.

Joins, and the check that belongs beside them

daily <- attendance |>
  left_join(roster, by = "student_id")

The failure to guard against is a join that multiplies rows because the right side has duplicate keys. dplyr warns, but a warning in a long script scrolls past:

before <- nrow(attendance)
daily <- attendance |> left_join(roster, by = "student_id")
stopifnot(nrow(daily) == before)

That assertion is the R equivalent of pandas’ validate="many_to_one", and it caught two students registered twice in the School attendance project — an unregistered transfer that left both registrations live.

anti_join() is the fastest way to see what did not match:

attendance |> anti_join(roster, by = "student_id") |> distinct(student_id)

An empty result means every attendance row found a student. A non-empty one is a list to take back to whoever maintains the roster.

What comes next

The verbs are in hand. The next lesson is the one they lead to and the one where most indicator bugs are born: group_by() and summarise(), and the three things R does with missing keys, unused levels and leftover grouping that pandas does the other way round.