---
title: "Computing FCS, HHS and rCSI from raw components"
subtitle: "Food security survey, 2024"
author: "Cassion · data-analysis.cassion.dev"
format:
  html:
    toc: true
    code-fold: false
jupyter: python3
---

## What this produces

The three composite food security indicators, built from their components rather
than read from a precomputed column — because the errors all happen in the
building.

Reference figures from the dataset's quality notes: on the standard 21/35 FCS
thresholds about 1% of households are poor and 23% borderline; on the 28/42 set,
7% and 39%. Neither is wrong. Failing to state which you used is.

Every dataset on this platform is synthetic. No real household is described.

## Setup

```{python}
import pandas as pd
import numpy as np

URL = (
    "https://data-analysis.cassion.dev/datasets/files/"
    "food-security-survey-2024.v1.csv"
)

fs = pd.read_csv(URL, dtype={"household_id": "string"})
print(fs.shape)
fs.head()
```

## Range-check before you score

Twenty-three records hold a consumption value above seven days, which is
impossible against a seven-day recall. A score computed without range-checking
inherits the impossible value and inflates that household — silently, because
the result is still a plausible number.

```{python}
FCS_WEIGHTS = {
    "fcs_cereals_tubers": 2,
    "fcs_pulses": 3,
    "fcs_vegetables": 1,
    "fcs_fruit": 1,
    "fcs_meat_fish_eggs": 4,
    "fcs_dairy": 4,
    "fcs_oils_fats": 0.5,
    "fcs_sugar": 0.5,
}

components = list(FCS_WEIGHTS)
impossible = (fs[components] > 7).sum().sum()
blank = fs[components].isna().sum().sum()

print(f"values above 7 days : {int(impossible)}")
print(f"blank cells         : {int(blank)}")
fs[components].agg(["min", "max"]).T
```

```{python}
# Out of range is not a measurement. Set it to missing rather than clipping to
# 7 — clipping invents a value the enumerator never recorded.
for column in components:
    fs[column] = fs[column].where(fs[column].between(0, 7))
```

## The blank that is not a zero

About 127 cells across dairy, fruit and meat are blank. **Treating a blank as
zero days scores the household as eating less than it did**, and the components
most often blank carry the heaviest weights — dairy and meat are 4 each. This is
the most consequential silent failure in this dataset, and it is one line of code
either way.

```{python}
fs["fcs_complete"] = fs[components].notna().all(axis=1)

# NaN propagates, so an incomplete household gets no score at all — which is the
# correct outcome. The zero-filled version is computed only to measure the harm.
fs["fcs"] = sum(fs[c] * w for c, w in FCS_WEIGHTS.items())
fs["fcs_zero_filled"] = sum(fs[c].fillna(0) * w for c, w in FCS_WEIGHTS.items())

incomplete = ~fs["fcs_complete"]
print(f"households with an incomplete FCS: {int(incomplete.sum())}")

pd.DataFrame({
    "complete households (real score)": fs.loc[~incomplete, "fcs"].describe(),
    "incomplete households, zero-filled": fs.loc[incomplete, "fcs_zero_filled"].describe(),
}).round(1)
```

The zero-filled households average about six points below the households that
answered fully. That gap is not a finding about their diet — it is the blanks
being counted as days of not eating.

```{python}
def share(scores, poor, borderline):
    s = scores.dropna()
    return {
        "poor %": round((s <= poor).mean() * 100, 2),
        "borderline %": round(((s > poor) & (s <= borderline)).mean() * 100, 2),
        "households": len(s),
    }

pd.DataFrame({
    "exclude incomplete (correct)": share(fs["fcs"], 21, 35),
    "zero-fill and keep": share(fs["fcs_zero_filled"], 21, 35),
}).T
```

At the 21/35 thresholds the distortion is small. At 28/42 it is not:

```{python}
zero_filled_incomplete = fs.loc[incomplete, "fcs_zero_filled"]
print(f"of the {len(zero_filled_incomplete)} incomplete households, zero-filling classifies")
print(f"  {int((zero_filled_incomplete <= 21).sum())} as poor at 21/35")
print(f"  {int((zero_filled_incomplete <= 28).sum())} as poor at 28/42")

pd.DataFrame({
    "exclude incomplete (correct)": share(fs["fcs"], 28, 42),
    "zero-fill and keep": share(fs["fcs_zero_filled"], 28, 42),
}).T
```

Thirty-two households classified as having poor food consumption on scores that
are artificially low, because a blank was read as a zero. Every one of them would
be counted in a caseload.

## Both threshold sets, side by side

```{python}
def consumption_group(score, poor, borderline):
    if pd.isna(score):
        return None
    if score <= poor:
        return "poor"
    if score <= borderline:
        return "borderline"
    return "acceptable"

valid = fs[fs["fcs_complete"]].copy()

for label, (poor, borderline) in {
    "21/35": (21, 35),
    "28/42": (28, 42),
}.items():
    valid[f"group {label}"] = valid["fcs"].apply(
        lambda s: consumption_group(s, poor, borderline)
    )

pd.DataFrame({
    label: valid[f"group {label}"].value_counts(normalize=True)
    for label in ["21/35", "28/42"]
}).round(3)
```

The 28/42 set is used where oil and sugar are consumed near-universally, which
inflates every score and makes the standard cut-offs too generous. Choosing it is
a judgement about the food system, not about the data — and **the choice moves
the headline from 1% poor to 7%**. State which set you used, in the same sentence
as the number.

## The Household Hunger Scale

HHS is valid only when all three questions are answered. A partial response must
be **excluded, not zero-filled** — zero-filling scores a hungry household as food
secure.

```{python}
HHS_ITEMS = [
    "hhs_no_food_in_house",
    "hhs_sleep_hungry",
    "hhs_day_and_night_without_eating",
]

complete_hhs = fs[HHS_ITEMS].notna().all(axis=1)
partial_hhs = fs[HHS_ITEMS].isna().any(axis=1) & fs[HHS_ITEMS].notna().any(axis=1)

print(f"complete: {int(complete_hhs.sum())}   partial: {int(partial_hhs.sum())}")

fs["hhs"] = fs[HHS_ITEMS].sum(axis=1).where(complete_hhs)
fs["hhs_category"] = pd.cut(
    fs["hhs"], [-1, 1, 3, 6],
    labels=["little to none", "moderate", "severe"],
)

(fs["hhs_category"].value_counts(normalize=True).sort_index() * 100).round(1)
```

Note `.sum(axis=1)` would happily return a score for a partial response — pandas
treats missing as zero in a row-wise sum. The `.where(complete_hhs)` is what
makes the exclusion real, and leaving it out is exactly the failure the note
warns about.

## The reduced Coping Strategies Index

```{python}
RCSI_WEIGHTS = {
    "rcsi_less_preferred_food": 1,
    "rcsi_borrowed_food": 2,
    "rcsi_limit_portion_size": 1,
    "rcsi_restrict_adult_consumption": 3,
    "rcsi_reduce_meal_numbers": 1,
}

fs["rcsi"] = sum(fs[c] * w for c, w in RCSI_WEIGHTS.items())
fs["rcsi"].describe().round(1)
```

The weights are severity, not frequency: restricting adult consumption so
children can eat scores 3, buying less preferred food scores 1. They are fixed by
the standard — do not adjust them to fit a context, because a locally-weighted
rCSI is not comparable to anything.

## Do not use one as a proxy for the other

```{python}
both = fs[fs["fcs_complete"]]
correlation = both[["fcs", "rcsi"]].corr().iloc[0, 1]
print(f"correlation between FCS and rCSI: {correlation:.3f}")
```

About -0.45. They measure related but distinct things: a household can eat
monotonously without yet resorting to coping strategies, and another can be
coping heavily while still eating a varied diet on borrowed food. Reporting one
as a stand-in for the other loses real information, and it is the kind of
shortcut that survives until someone asks why the two tables disagree.

```{python}
pd.crosstab(
    both["fcs"].apply(lambda s: consumption_group(s, 21, 35)),
    pd.cut(both["rcsi"], [-1, 3, 18, 100], labels=["low", "medium", "high"]),
    normalize="index",
).round(3)
```

## What this does and does not produce

These are the food consumption evidence used **in** an IPC analysis. They are not
an IPC phase. A phase is assigned by a technical working group convening several
outcome indicators against contributing factors, and a table that prints "Phase
3" out of an FCS distribution has skipped the entire analytical process the
classification exists to represent.