Back to the lessonLesson 3 of 8Data that keeps its meaning

Stata and SPSS files keep their labels

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDF Slides · PowerPoint

Slide 1 / 23
What this lesson covers
- What a CSV throws away
- The labelled class
- Converting deliberately
- The two ways to lose a codebook
- User-defined missing values
- SPSS and the other formats
- Where this fits with the Python course
- What comes next
Speaker notes
haven, the labelled class, and why a survey firm's .dta should be read in R even when the analysis happens elsewhere — plus the two ways to lose a codebook without noticing.
Slide 2 / 23
What a CSV throws away — In R
```
library(haven)

survey <- read_dta(here("data", "raw", "household-survey.dta"))
```
Speaker notes
A survey firm delivers a .dta or a .sav. Inside it, a column is not just values — it carries a variable label ("Household head consented to interview") and value labels (1 = Yes, 2 = No, 9 = Don't know). Export it to CSV and both are gone. You get a column of 1, 2 and 9, a separate PDF codebook somebody has to keep, and a mean of 1.7 that means nothing at all. haven reads the labels along with the values. That is the reason this lesson exists, and the reason to read a survey file in R even when the analysis will happen in Python: R preserves more of the file than pandas does, and a CSV plus a codebook you wrote yourself is a worse artefact than the original.

Slide 3 / 23

The `labelled` class — In R

class(survey$consent)
#> [1] "haven_labelled" "vctrs_vctr" "double"

attr(survey$consent, "labels")
#>       Yes        No Don't know
#>         1         2          9

attr(survey$consent, "label")
#> [1] "Household head consented to interview"

Speaker notes

read_dta() gives you columns of class haven_labelled: the underlying values, with their labels attached.

Slide 4 / 23
The labelled class — In R
```
mean(survey$consent, na.rm = TRUE)
#> [1] 1.7
```
Speaker notes
The values are still there — arithmetic works — which is exactly the trap. A labelled column is a number as far as mean() is concerned: That figure is the average of 1, 2 and 9. It is meaningless and it does not warn.
Slide 5 / 23
Converting deliberately
- A categorical variable becomes a factor
Speaker notes
haven gives you three moves, and which one is right depends on what the column is. A categorical variable becomes a factor.

Slide 6 / 23

Converting deliberately — In R

library(dplyr)

survey <- survey |>
  mutate(consent = as_factor(consent))

levels(survey$consent)
#> [1] "Yes" "No" "Don't know"

Slide 7 / 23
Converting deliberately
- A genuine number loses its labels
Speaker notes
as_factor() — haven's, not base R's as.factor() — uses the value labels as levels, in the order the file declares them rather than alphabetically. That ordering is a gift: the codebook's order is usually the order a report wants. A genuine number loses its labels.

Slide 8 / 23

Converting deliberately — In R

survey <- survey |> mutate(hh_size = zap_labels(hh_size))

Slide 9 / 23
Converting deliberately
- Everything at once, when the file is uniformly categorical
Speaker notes
zap_labels() strips the labelling and leaves the values. Use it where the column really is a quantity that happened to carry a label. Everything at once, when the file is uniformly categorical:
Slide 10 / 23
Converting deliberately — In R
```
survey <- read_dta(path) |> as_factor()
```
Speaker notes
Applying as_factor() to the whole tibble converts every labelled column. Fast, and wrong for any file mixing categories with quantities — check with glimpse() before reaching for it.
Slide 11 / 23
The two ways to lose a codebook
- Reading with the labels already applied

Slide 12 / 23

The two ways to lose a codebook — In R

survey <- read_dta(path, .name_repair = "unique")
survey <- as_factor(survey)          # values are gone; only labels remain

Slide 13 / 23
The two ways to lose a codebook — In R
```
survey <- read_dta(path)

codes <- survey |> mutate(across(where(is.labelled), zap_labels))
labels <- survey |> as_factor()
```
Speaker notes
Now the underlying codes are unavailable. That matters more than it sounds: the codebook, the questionnaire, the tabulation plan and every previous analysis all refer to 1 and 2. A colleague asking "how many were coded 9?" cannot be answered from a factor. Keep both:
Slide 14 / 23
The two ways to lose a codebook
- Writing the CSV and moving on
Slide 15 / 23
The two ways to lose a codebook — In R
```
readr::write_csv(as_factor(survey), "survey.csv")
```
Slide 16 / 23
The two ways to lose a codebook — In R
```
codebook <- tibble::tibble(
  variable = names(survey),
  label    = vapply(survey, function(x) attr(x, "label") %||% NA_character_, character(1))
)

readr::write_csv(codebook, "survey-codebook.csv")
```
Speaker notes
The CSV now holds "Yes" and "No", which is readable and lossy in a different direction — the codes are gone, and so is the distinction between "Don't know" and a genuine missing value. If you must hand a CSV to a Python analysis, hand a codebook with it, generated from the file rather than typed: The value labels need one row per level, which labelled::look_for() produces directly if you have that package. Either way, the codebook is derived, not transcribed — a transcribed codebook drifts from the file it describes.
Slide 17 / 23
User-defined missing values — In R
```
survey$income
#> <labelled<double>[3]>
#> [1]  4500 NA(a) NA(b)
```
Speaker notes
Stata and SPSS distinguish kinds of missing: refused, not applicable, not asked. haven preserves them as tagged NAs rather than collapsing them.
Slide 18 / 23
User-defined missing values — In R
```
survey <- survey |> mutate(income = zap_missing(income))
```
Speaker notes
NA(a) and NA(b) are still NA to is.na(), so nothing silently becomes a number. But the distinction survives, and it is often the finding: a question refused by 40% of respondents is a different problem from one that did not apply to them. zap_missing() flattens them to plain NA once you have counted them. Count first.

Slide 19 / 23

SPSS and the other formats — In R

read_sav(path)     # SPSS .sav
read_por(path)     # SPSS portable
read_dta(path)     # Stata
read_xpt(path)     # SAS transport

Slide 20 / 23
SPSS and the other formats — In R
```
write_dta(survey, "cleaned.dta")
```
Speaker notes
All return the same labelled structure, so everything above applies unchanged. Writing back is symmetrical, and occasionally necessary when a ministry expects a .dta: Stata has variable-name rules R does not — no dots, 32 characters — and write_dta() will error rather than silently mangle. That is the behaviour you want.
Slide 21 / 23
Where this fits with the Python course
- The Python for Programme Data course says, in its lesson on codes and sentinels, that this is the one place where the answer is "use the other language".
Speaker notes
The Python for Programme Data course says, in its lesson on codes and sentinels, that this is the one place where the answer is "use the other language". This is the lesson it points at. pandas reads a .dta and gives you either the codes or the labels, not both, and does not preserve tagged missing values at all. If the labels matter — and on a survey file they always do — read in R, convert deliberately, and export both the codes and a derived codebook.
Slide 22 / 23
What comes next
- The labels are attached and converted into factors.
Speaker notes
The labels are attached and converted into factors. The next lesson is about what a factor actually is in R, and about the ordering problem as_factor() solved for you here and will not solve for a column you build yourself.
Slide 23 / 23
Where this goes next
Read the full lesson, with runnable code Back to the lesson

What this lesson covers

What a CSV throws away — In R

The labelled class — In R

The labelled class — In R

Converting deliberately

Converting deliberately — In R

Converting deliberately

Converting deliberately — In R

Converting deliberately

Converting deliberately — In R

The two ways to lose a codebook

The two ways to lose a codebook — In R

The two ways to lose a codebook — In R

The two ways to lose a codebook

The two ways to lose a codebook — In R

The two ways to lose a codebook — In R

User-defined missing values — In R

User-defined missing values — In R

SPSS and the other formats — In R

SPSS and the other formats — In R

Where this fits with the Python course

What comes next

Where this goes next

The `labelled` class — In R

The `labelled` class — In R