Factors, and the order a report needs

A factor is an integer with a lookup table

service <- factor(c("basic", "limited", "unimproved", "basic"))

levels(service)
#> [1] "basic"      "limited"    "unimproved"

as.integer(service)
#> [1] 1 2 3 1

Underneath, R stores 1 2 3 1 and a character vector of levels. Everything that follows — the ordering problem, the as.numeric() trap, unused levels — comes from that one fact.

Notice the levels are alphabetical. Nothing about basic, limited, unimproved says that is the order, and it is exactly backwards for a JMP service ladder, where unimproved is worst and basic is better than limited. Left alone, every table and every chart built from that column reads in an order that means nothing.

The as.numeric() trap

This is the single most expensive factor mistake, and it does not warn.

age <- factor(c("10", "9", "8"))

as.numeric(age)
#> [1] 1 3 2

Those are level positions, not values. The levels are alphabetical — "10", "8", "9" — so "10" is position 1 and "9" is position 3.

as.numeric(as.character(age))
#> [1] 10 9 8

as.character() first, always. It arises whenever a numeric column was read as text — an Excel column with one stray "n/a" in it, a CSV read with col_character() — and converted to a factor along the way.

Watch for a mean that is suspiciously close to the number of categories.

Setting the order you want

library(forcats)

ladder <- fct_relevel(service, "unimproved", "limited", "basic")
levels(ladder)
#> [1] "unimproved" "limited"    "basic"

fct_relevel() names the order explicitly. Declare it once, near where the column is created, and every table, chart and model downstream inherits it.

Two forcats verbs earn their place immediately:

fct_infreq(source)      # most common level first
fct_reorder(name, value)  # order one factor by another column

fct_reorder() is what turns an unreadable bar chart into a readable one — the communes sorted by their GAM rate rather than by the alphabet — and it does it without a manual levels = you have to update when the data changes.

library(dplyr)

by_commune |>
  mutate(commune = fct_reorder(commune, gam_rate)) |>
  ggplot2::ggplot(ggplot2::aes(gam_rate, commune)) +
  ggplot2::geom_col()

Ordered factors

fct_relevel() sets the order of the levels. It does not make the factor comparable:

service[1] >= service[2]
#> Warning: '>=' not meaningful for factors
#> [1] NA

For a ladder, where “at least basic” is a real question, declare it ordered:

ladder <- factor(
  c("basic", "limited"),
  levels = c("unimproved", "limited", "basic"),
  ordered = TRUE
)

ladder[1] >= ladder[2]
#> [1] TRUE

Now >= "basic" computes the JMP coverage indicator directly. Without ordered = TRUE it returns NA with a warning, which — if the warning is not read — becomes a coverage figure of NA or, worse, a filter that selects nothing.

Use ordered factors sparingly. They change how models treat the variable (polynomial contrasts rather than dummies), which is rarely what you want in a regression. For a service ladder, a severity band or an IPC phase, the ordering is worth it. For a commune, it is not.

Unused levels, and the two defaults that disagree

A level with no rows is still a level. Whether it appears in your output depends on which function you ask, and R is not internally consistent about it.

d <- tibble::tibble(k = factor(c("a", "b", "a"), levels = c("a", "b", "c")))

nrow(dplyr::count(d, k))
#> [1] 2

length(table(d$k))
#> [1] 3

count() drops the empty level; table() keeps it. Both are right for different questions, and the question is the same one the Python for Programme Data course raises about observed=: reporting on facilities that submitted data wants the empty one gone, and reporting coverage against a list of facilities that should have submitted wants it there, because a facility with zero rows is the finding.

Say which you mean:

dplyr::count(d, k, .drop = FALSE)   # keep the empty level
droplevels(d$k)                     # remove levels with no rows
forcats::fct_drop(d$k)              # the forcats spelling of the same thing

Collapsing a long tail

Ten water sources is too many rows for a report table and too many colours for a chart.

source <- factor(wash$water_source)
length(levels(source))
#> [1] 10

levels(fct_lump_n(source, 4))
#> [1] "borehole" "piped-into-dwelling" "piped-into-yard" "public-tap" "Other"

fct_lump_n() keeps the four most common and folds the rest into Other.

Do not lump before computing an indicator. Other here mixes protected springs, which are improved, with surface water, which is not — so a coverage rate computed after lumping is wrong. Lump for presentation, on a copy, after the numbers are settled.

fct_recode(source,
  improved = "borehole",
  improved = "protected-well",
  unimproved = "surface-water"
)

fct_recode() is the explicit alternative and is what you want for a classification: it names every mapping, so a source that appears in the next export and is not in the list stays itself rather than silently joining a group.

Factors and missing values

NA is not a level, so it survives every reordering and appears in table() only if you ask:

table(muac$outcome, useNA = "ifany")

To make missingness an explicit category — which is often right in a report, where “not recorded” is a finding rather than a gap:

muac <- muac |> mutate(outcome = fct_na_value_to_level(outcome, level = "not recorded"))

Once it is a level it will be counted, plotted and summed like any other. That is the point, and it is also the risk: a rate computed over a factor that includes “not recorded” has a different denominator from one that does not.

Where factors bite in a join

left_join(muac, sites, by = "commune")

If commune is a factor in one frame and a character in the other, dplyr coerces and warns. If it is a factor in both with different levels, the join still works — dplyr compares the labels, not the integers — but the result’s levels are the union, which can quietly reintroduce empty categories.

The simplest habit: join on character columns, convert to factor afterwards. Factors are a presentation and modelling device, not a key type.

What comes next

Columns now hold their meaning and their order. The next unit works them: the dplyr verbs, and then the grouping call where most indicator bugs are born.