Back to the lessonLesson 5 of 8The verbs

The dplyr verbs, used deliberately

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDF Slides · PowerPoint

Slide 1 / 23
What this lesson covers
- Five verbs and a pipe
- filter() drops missing values, silently
- select() and the helpers
- mutate() and the two conditional functions
- across(): one rule, many columns
- arrange()
- distinct() and counting
- Joins, and the check that belongs beside them
- What comes next
Speaker notes
filter, select, mutate and arrange — what each does with a missing value, why if_else() refuses what ifelse() silently accepts, and the across() that applies one rule to many columns.
Slide 2 / 23
Five verbs and a pipe — In R
```
library(dplyr)

muac |>
  filter(!is.na(muac_mm)) |>
  mutate(gam = muac_mm < 125) |>
  group_by(commune) |>
  summarise(assessed = n(), cases = sum(gam)) |>
  arrange(desc(cases / assessed))
```
Speaker notes
Each verb takes a data frame and returns one. The pipe passes the result along. That is the whole grammar, and the rest of this lesson is about what each verb does when the data is not clean — which is always. |> is R's native pipe, available since 4.1. %>% from magrittr predates it and behaves near-identically for this kind of code; if your team's scripts use it, everything here works unchanged.

Slide 3 / 23

`filter()` drops missing values, silently — In R

nrow(muac)
#> [1] 4218

nrow(filter(muac, muac_mm < 125))
#> [1] 319

sum(is.na(muac$muac_mm))
#> [1] 72

Slide 4 / 23
filter() drops missing values, silently — In R
```
# Children measured below 125 mm.
filter(muac, muac_mm < 125)

# Children not known to be at or above 125 mm — the unmeasured included.
filter(muac, is.na(muac_mm) | muac_mm < 125)
#> 391 rows
```
Speaker notes
A comparison against NA yields NA, and filter() keeps only rows that are TRUE. So those 72 unmeasured children are gone, and nothing said so. This is the same behaviour as pandas, and it is worth stating in both languages because it is the fastest way to change a denominator by accident. Be explicit about which you meant: Seventy-two rows separate those two questions. Which one you want depends on whether an unmeasured child is a non-case or an unknown, and that is a decision for a cleaning log, not a default.

Slide 5 / 23

`select()` and the helpers — In R

muac |> select(child_id, commune, muac_mm)
muac |> select(-oedema)
muac |> select(starts_with("muac"), ends_with("_date"))
muac |> select(where(is.numeric))

Slide 6 / 23
select() and the helpers — In R
```
epi |> select(facility = facility_id, month = period, doses = doses_administered)
```
Speaker notes
select() renames as it selects, which is the tidy way to fix an export's column names once at the top: Two habits worth forming. Select before you join, so the result does not carry forty columns you did not want. And never select by position — select(3:5) breaks the day the export gains a column, and it breaks silently because the code still runs.

Slide 7 / 23

`mutate()` and the two conditional functions — In R

muac <- muac |>
  mutate(
    gam = muac_mm < 125,
    sam = muac_mm < 115,
    band = case_when(
      muac_mm < 115 ~ "severe",
      muac_mm < 125 ~ "moderate",
      .default = "normal"
    )
  )

Slide 8 / 23
mutate() and the two conditional functions — In R
```
case_when(c(110, 130, NA) < 125 ~ "case", .default = "not")
#> [1] "case" "not"  "not"
```
Speaker notes
case_when() evaluates in order and the first match wins, so < 115 must come first. Its .default catches everything else including NA, which is almost never what you want:
Slide 9 / 23
mutate() and the two conditional functions — In R
```
band = case_when(
  is.na(muac_mm) ~ NA_character_,
  muac_mm < 115  ~ "severe",
  muac_mm < 125  ~ "moderate",
  .default = "normal"
)
```
Speaker notes
The missing measurement became "not". Handle it explicitly:
Slide 10 / 23
mutate() and the two conditional functions
- if_else() rather than ifelse()
Speaker notes
Putting the is.na() branch first is the habit. This is the same trap as np.select's default in the Python course, and it is worth recognising in both places.
Slide 11 / 23
mutate() and the two conditional functions — In R
```
ifelse(c(TRUE, FALSE), 1L, "x")
#> [1] "1" "x"
```
Slide 12 / 23
mutate() and the two conditional functions — In R
```
if_else(c(TRUE, FALSE), 1L, "x")
#> Error: Can't combine `true` <integer> and `false` <character>.
```
Speaker notes
Base R's ifelse() silently coerced an integer and a string into a character vector. A column you believed was numeric is now text, and you find out three steps later when a sum fails. dplyr's if_else() refuses. Prefer it everywhere, for the reason this course keeps returning to: an error where the mistake is made beats a wrong answer three steps later. if_else() also takes a missing = argument, which is how you say what an NA condition should produce rather than letting it propagate.

Slide 13 / 23

`across()`: one rule, many columns — In R

muac |>
  summarise(across(c(muac_mm, age_months), ~ mean(.x, na.rm = TRUE)))
#> # A tibble: 1 × 2
#>   muac_mm age_months
#> 1    140.       30.9

Slide 14 / 23

`across()`: one rule, many columns — In R

survey |> mutate(across(where(is.character), stringr::str_squish))

survey |> mutate(across(starts_with("fcs_"), ~ if_else(.x > 7, NA_real_, .x)))

Speaker notes

across() is what stops a cleaning script being fifteen near-identical lines.

Slide 15 / 23
across(): one rule, many columns — In R
```
survey |> mutate(across(starts_with("fcs_"), \(x) if_else(x > 7, NA_real_, x)))
```
Speaker notes
That second line is the food-security rule from the Python for Programme Data course, in one statement: a consumption value above seven days is impossible and must become missing rather than being clipped, across all eight food groups at once. The ~ .x form is a shorthand for a function of one argument. \(x) ... is the newer base R spelling and works identically:
Slide 16 / 23
arrange() — In R
```
muac |> arrange(commune, desc(screening_date))
```
Speaker notes
NA sorts last regardless of direction, which is usually what you want and occasionally hides a problem — a column that is entirely missing looks sorted. Sorting arranges rows. It says nothing about whether the differences between adjacent rows are real, a point the Nutrition programme dashboard project makes at length where nine of twelve communes have overlapping confidence intervals.

Slide 17 / 23

`distinct()` and counting — In R

n_distinct(muac$commune)
#> [1] 12

muac |> distinct(child_id, .keep_all = TRUE)

Slide 18 / 23
distinct() and counting — In R
```
muac |> count(commune, sort = TRUE)
```
Speaker notes
distinct() without .keep_all = TRUE returns only the named columns, which surprises people expecting deduplicated rows. With it, the first row of each group is kept — so sort first if which one survives matters. count() is group_by() |> summarise(n = n()) |> ungroup(), and it is the right tool whenever that is all you want.
Slide 19 / 23
Joins, and the check that belongs beside them — In R
```
daily <- attendance |>
  left_join(roster, by = "student_id")
```
Slide 20 / 23
Joins, and the check that belongs beside them — In R
```
before <- nrow(attendance)
daily <- attendance |> left_join(roster, by = "student_id")
stopifnot(nrow(daily) == before)
```
Speaker notes
The failure to guard against is a join that multiplies rows because the right side has duplicate keys. dplyr warns, but a warning in a long script scrolls past:
Slide 21 / 23
Joins, and the check that belongs beside them — In R
```
attendance |> anti_join(roster, by = "student_id") |> distinct(student_id)
```
Speaker notes
That assertion is the R equivalent of pandas' validate="many_to_one", and it caught two students registered twice in the School attendance project — an unregistered transfer that left both registrations live. anti_join() is the fastest way to see what did not match: An empty result means every attendance row found a student. A non-empty one is a list to take back to whoever maintains the roster.
Slide 22 / 23
What comes next
- The verbs are in hand.
Speaker notes
The verbs are in hand. The next lesson is the one they lead to and the one where most indicator bugs are born: group_by() and summarise(), and the three things R does with missing keys, unused levels and leftover grouping that pandas does the other way round.
Slide 23 / 23
Where this goes next
Read the full lesson, with runnable code Back to the lesson

What this lesson covers

Five verbs and a pipe — In R

filter() drops missing values, silently — In R

filter() drops missing values, silently — In R

select() and the helpers — In R

select() and the helpers — In R

mutate() and the two conditional functions — In R

mutate() and the two conditional functions — In R

mutate() and the two conditional functions — In R

mutate() and the two conditional functions

mutate() and the two conditional functions — In R

mutate() and the two conditional functions — In R

across(): one rule, many columns — In R

across(): one rule, many columns — In R

across(): one rule, many columns — In R

arrange() — In R

distinct() and counting — In R

distinct() and counting — In R

Joins, and the check that belongs beside them — In R

Joins, and the check that belongs beside them — In R

Joins, and the check that belongs beside them — In R

What comes next

Where this goes next

`filter()` drops missing values, silently — In R

`filter()` drops missing values, silently — In R

`select()` and the helpers — In R

`select()` and the helpers — In R

`mutate()` and the two conditional functions — In R

`mutate()` and the two conditional functions — In R

`mutate()` and the two conditional functions — In R

`mutate()` and the two conditional functions

`mutate()` and the two conditional functions — In R

`mutate()` and the two conditional functions — In R

`across()`: one rule, many columns — In R

`across()`: one rule, many columns — In R

`across()`: one rule, many columns — In R

`arrange()` — In R

`distinct()` and counting — In R

`distinct()` and counting — In R