Stata and SPSS files keep their labels

What a CSV throws away

A survey firm delivers a .dta or a .sav. Inside it, a column is not just values — it carries a variable label (“Household head consented to interview”) and value labels (1 = Yes, 2 = No, 9 = Don't know).

Export it to CSV and both are gone. You get a column of 1, 2 and 9, a separate PDF codebook somebody has to keep, and a mean of 1.7 that means nothing at all.

library(haven)

survey <- read_dta(here("data", "raw", "household-survey.dta"))

haven reads the labels along with the values. That is the reason this lesson exists, and the reason to read a survey file in R even when the analysis will happen in Python: R preserves more of the file than pandas does, and a CSV plus a codebook you wrote yourself is a worse artefact than the original.

The labelled class

read_dta() gives you columns of class haven_labelled: the underlying values, with their labels attached.

class(survey$consent)
#> [1] "haven_labelled" "vctrs_vctr" "double"

attr(survey$consent, "labels")
#>       Yes        No Don't know
#>         1         2          9

attr(survey$consent, "label")
#> [1] "Household head consented to interview"

The values are still there — arithmetic works — which is exactly the trap. A labelled column is a number as far as mean() is concerned:

mean(survey$consent, na.rm = TRUE)
#> [1] 1.7

That figure is the average of 1, 2 and 9. It is meaningless and it does not warn.

Converting deliberately

haven gives you three moves, and which one is right depends on what the column is.

A categorical variable becomes a factor.

library(dplyr)

survey <- survey |>
  mutate(consent = as_factor(consent))

levels(survey$consent)
#> [1] "Yes" "No" "Don't know"

as_factor() — haven’s, not base R’s as.factor() — uses the value labels as levels, in the order the file declares them rather than alphabetically. That ordering is a gift: the codebook’s order is usually the order a report wants.

A genuine number loses its labels.

survey <- survey |> mutate(hh_size = zap_labels(hh_size))

zap_labels() strips the labelling and leaves the values. Use it where the column really is a quantity that happened to carry a label.

Everything at once, when the file is uniformly categorical:

survey <- read_dta(path) |> as_factor()

Applying as_factor() to the whole tibble converts every labelled column. Fast, and wrong for any file mixing categories with quantities — check with glimpse() before reaching for it.

The two ways to lose a codebook

Reading with the labels already applied

survey <- read_dta(path, .name_repair = "unique")
survey <- as_factor(survey)          # values are gone; only labels remain

Now the underlying codes are unavailable. That matters more than it sounds: the codebook, the questionnaire, the tabulation plan and every previous analysis all refer to 1 and 2. A colleague asking “how many were coded 9?” cannot be answered from a factor.

Keep both:

survey <- read_dta(path)

codes <- survey |> mutate(across(where(is.labelled), zap_labels))
labels <- survey |> as_factor()

Writing the CSV and moving on

readr::write_csv(as_factor(survey), "survey.csv")

The CSV now holds "Yes" and "No", which is readable and lossy in a different direction — the codes are gone, and so is the distinction between “Don’t know” and a genuine missing value.

If you must hand a CSV to a Python analysis, hand a codebook with it, generated from the file rather than typed:

codebook <- tibble::tibble(
  variable = names(survey),
  label    = vapply(survey, function(x) attr(x, "label") %||% NA_character_, character(1))
)

readr::write_csv(codebook, "survey-codebook.csv")

The value labels need one row per level, which labelled::look_for() produces directly if you have that package. Either way, the codebook is derived, not transcribed — a transcribed codebook drifts from the file it describes.

User-defined missing values

Stata and SPSS distinguish kinds of missing: refused, not applicable, not asked. haven preserves them as tagged NAs rather than collapsing them.

survey$income
#> <labelled<double>[3]>
#> [1]  4500 NA(a) NA(b)

NA(a) and NA(b) are still NA to is.na(), so nothing silently becomes a number. But the distinction survives, and it is often the finding: a question refused by 40% of respondents is a different problem from one that did not apply to them.

survey <- survey |> mutate(income = zap_missing(income))

zap_missing() flattens them to plain NA once you have counted them. Count first.

SPSS and the other formats

read_sav(path)     # SPSS .sav
read_por(path)     # SPSS portable
read_dta(path)     # Stata
read_xpt(path)     # SAS transport

All return the same labelled structure, so everything above applies unchanged.

Writing back is symmetrical, and occasionally necessary when a ministry expects a .dta:

write_dta(survey, "cleaned.dta")

Stata has variable-name rules R does not — no dots, 32 characters — and write_dta() will error rather than silently mangle. That is the behaviour you want.

Where this fits with the Python course

The Python for Programme Data course says, in its lesson on codes and sentinels, that this is the one place where the answer is “use the other language”. This is the lesson it points at.

pandas reads a .dta and gives you either the codes or the labels, not both, and does not preserve tagged missing values at all. If the labels matter — and on a survey file they always do — read in R, convert deliberately, and export both the codes and a derived codebook.

What comes next

The labels are attached and converted into factors. The next lesson is about what a factor actually is in R, and about the ordering problem as_factor() solved for you here and will not solve for a column you build yourself.