# Plausibility report — SMART survey, 30 clusters, 2024 Technical documentation for the project *SMART survey analysis and plausibility report*. This is the document the publication decision rests on: the prevalence estimate is the easy half, and a SMART estimate is not reportable until the plausibility checks have been run and their verdicts stated. - **Source dataset:** `smart-nutrition-survey-2024.v1.csv` (930 children, 30 clusters, 4 measurement teams) - **Reference table:** `who-2006-weight-for-lenhei.csv` (WHO 2006 LMS parameters, exported from the official `anthro` R package) - **Analyses:** `notebooks/whz-prevalence.python.en.ipynb`; the check-by-check working is in the dataset example `plausibility.python` - **Decision informed:** whether to accept the survey, exclude a measurement team's data, or repeat the fieldwork - **Standards applied:** SMART methodology, WHO growth standards, UNICEF indicator definitions Every dataset on this platform is synthetic. No real child is described, and these results must not be cited as a real nutrition situation. --- ## Verdict **Do not publish as measured. Re-measure team 3's clusters if the teams are still in the field; if they are not, publish with the team comparison in the body of the report rather than an annex.** Two checks fail and two warn: | # | Check | Value | Result | | ---: | --- | --- | --- | | 1 | Impossible measurements | 1.5% excluded | pass | | 2 | Flagged records (SMART, ±3 SD from survey mean) | 0.3% | pass | | 3 | SD of weight-for-height z | 1.23 | **warning** | | 4 | Digit preference | team 2 at 69% on .0/.5 | **fail** | | 5 | Age heaping | 24% on whole years | **warning** | | 6 | Sex ratio | 1.01 | pass | | 7 | Between-team bias | 0.67 z spread | **fail** | The two failures are in **different teams**. The survey has two independent measurement faults, and only one of them moves the prevalence estimate. --- ## 1. The estimate, and why it cannot be published as it stands | | Value | | --- | --- | | Children analysable | 872 | | GAM (WHZ < −2 or oedema) | **14.9%** | | 95% CI | 11.3% – 18.5% | | SAM (WHZ < −3 or oedema) | 3.9% | | Design effect | 2.28 | | Mean WHZ | −0.673 | | SD WHZ | 1.228 | The emergency threshold that would trigger a scale-up is 15%. **The point estimate sits just below it and the interval crosses it.** On that alone the survey cannot say whether the scale-up criterion is met — which is why the team question in §4 has to be settled before the number is used at all. ### 1.1 How the interval is computed Clusters are the sampling unit, so the variance must be computed between clusters, not between children. The ultimate-cluster ratio estimator: ``` p = Σ yᵢ / Σ mᵢ var = k / ((k−1) · M²) · Σ (yᵢ − p·mᵢ)² ``` with `k` = 30 clusters, `mᵢ` children analysable in cluster *i*, `yᵢ` cases in cluster *i*, `M = Σ mᵢ`. The 95% interval is `p ± 1.96·√var`. Design effect is that variance over the simple-random-sampling variance `p(1−p)/M`: **2.28**, implying an intracluster correlation near 0.045. Treating the 872 children as a simple random sample would report an interval roughly √2.28 ≈ 1.5 times too narrow, and would place the whole interval below the threshold. ### 1.2 Z-score computation Weight-for-height z-scores are computed against the WHO 2006 LMS parameters, not read from a pre-computed column: - Length is used below 24 months, height at 24 months and above. Where the recorded measurement position disagrees with the age rule, the measurement is converted by 0.7 cm. - Beyond ±3 SD, WHO replaces the LMS curve with a linear extension based on the distance between the 2 and 3 SD cut-offs. This is applied; omitting it inflates the tails, which is exactly where GAM and SAM are counted. - Oedema overrides anthropometry: a child with bilateral pitting oedema is a GAM and SAM case whatever the z-score. The reference table is shipped with this platform because R has the official WHO package and Python has no maintained equivalent that installs cleanly. The R and Python analyses agree on prevalence to the first decimal. --- ## 2. Exclusions, before any check is run | Category | Children | Treatment | | --- | ---: | --- | | Impossible measurement (weight outside 2–30 kg, height outside 45–130 cm) | 14 | Excluded — data entry error | | Missing weight | 11 | Kept in the file, scored NA | | Missing age | 21 | Kept in the file, scored NA | | Analysable | 872 | — | **A missing measurement and an impossible one are different failures and only one of them is a data-entry error.** In pandas, `between()` returns False for NaN, so a bare `~between(...)` counts a missing measurement as impossible — an error that inflated the exclusion count from 14 to 25 in an earlier draft of this analysis. It is recorded here because it is invisible in the output. --- ## 3. Checks 1–6, in detail ### Check 1 — Impossible values · **pass** 14 of 930 (1.5%) excluded. Below the level at which SMART would question the fieldwork as a whole. ### Check 2 — Flagged records · **pass** Applying the SMART flag (±3 SD from the *survey* mean rather than the WHO fixed range): 3 records, 0.33%. The WHO fixed range (−5 to +5) flags 1. Both are far below the SMART concern level. The SMART flag is relative to the survey mean, so it identifies records inconsistent with *this* survey; the WHO flag identifies records inconsistent with the growth standard. They answer different questions and both are reported. ### Check 3 — Standard deviation of WHZ · **warning** **1.228.** SMART treats 0.8–1.2 as acceptable and above 1.2 as a sign of measurement imprecision or a genuinely heterogeneous population. At 1.23 this is marginal, and §4 supplies the likely mechanism: a single team measuring systematically low widens the pooled distribution. Excluding team 3 would test that directly. ### Check 4 — Digit preference · **fail** Share of height measurements ending in `.0` or `.5`, expected 20% under no preference: | Team | % on .0/.5 | | --- | ---: | | 1 | 17.6% | | **2** | **69.4%** | | 3 | 18.0% | | 4 | 23.4% | Team 2's full final-digit distribution puts 34.7% on `.0` and 34.7% on `.5`, against 2.5–5.4% on every other digit. This is not rounding noise; it is a team reading the board to the nearest half-centimetre. **This is a fail, and it names a different team from check 7.** Team 2's mean z-score is −0.427, the second *highest* of the four — its measurement fault adds noise rather than bias, so it does not move the prevalence point estimate. It does inflate the SD (check 3) and it is a training finding regardless. ### Check 5 — Age heaping · **warning** **211 children (23.6%) at an exact whole year**, with pile-ups at 24, 36 and 48 months (65, 80 and 44 children against 10–26 at neighbouring months). This matters here for one specific reason: **the length/height rule switches at exactly 24 months.** A child heaped onto 24 months is measured standing and compared against the height standard when they may be 22 months old. It is a second, smaller source of error running through every team. ### Check 6 — Sex ratio · **pass** 461 boys, 455 girls; ratio 1.013, chi-square against 1:1 = 0.04. No evidence of selective inclusion. --- ## 4. Check 7 — Between-team bias · **fail** | Team | Children | Mean WHZ | SD | Mean height (cm) | Mean weight (kg) | GAM | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 1 | 208 | −0.687 | 1.240 | 76.11 | 9.07 | 15.9% | | 2 | 232 | −0.427 | 1.173 | 75.97 | 9.26 | 11.2% | | **3** | **230** | **−1.099** | 1.224 | 75.24 | **8.59** | **22.2%** | | 4 | 202 | −0.457 | 1.155 | 75.41 | 9.04 | 9.9% | **Spread in mean z-score: 0.67.** SMART treats a between-team spread above roughly 0.3 z as a supervision problem. **Clusters were assigned to teams independently of nutrition status**, so a real difference of two thirds of a z-score between teams would be extraordinary. Team 3 measures children 0.5 kg lighter on average than the other three at almost the same mean height — the signature of a mis-set or drifting scale rather than a poorer population. Team 3's digit preference (18.0%) is unremarkable, so this is not the same fault as check 4. Two teams, two independent faults. --- ## 5. The three options, costed | Option | GAM | 95% CI | DEFF | n | Crosses 15% | | --- | ---: | --- | ---: | ---: | --- | | Accept as measured | 14.9% | 11.3 – 18.5% | 2.28 | 872 | yes | | Exclude team 3 | 12.3% | 8.6 – 16.0% | 2.16 | 642 | yes | | Re-measure team 3's clusters | — | — | — | — | — | **Difference: −2.6 points.** The scale-up decision turns on exactly that difference. **Do not accept as measured.** The estimate is inflated by a team whose mean z-score sits two thirds of a standard deviation below the others across independently assigned clusters. **Do not silently exclude the team either.** Excluding 8 of 30 clusters changes the sampling frame, and a prevalence computed on the remainder is no longer the survey that was designed. Note that the interval still crosses the threshold, so exclusion does not even resolve the question it was meant to resolve. It is reported as a sensitivity analysis, not as the answer. **Re-measure.** If the teams are still in the field, re-measuring team 3's eight clusters is the only option that produces a publishable estimate. If they are not, publish with §4 in the body of the report. A reader shown only the district figure cannot know that a quarter of the sample was measured by someone whose results match nobody else's. --- ## 6. What this report does not establish 1. **Which team is right.** The comparison shows one team differs; it does not show the other three are correct. If a standardisation exercise was run before fieldwork its results settle this, and this analysis does not have them. 2. **Whether team 3's clusters are genuinely worse.** Independent assignment makes it unlikely but not impossible. Only re-measurement separates the two. 3. **The cause of the digit preference or the scale bias.** A supervision visit answers that; a dataset does not. 4. **Anything about MUAC-based case-finding.** MUAC and weight-for-height identify overlapping but different children, and a GAM by WHZ is not comparable to a GAM by MUAC — see the *Nutrition programme dashboard* project. --- ## 7. Reproducing this report ```bash pnpm examples:build ``` Then run `notebooks/whz-prevalence.python.en.ipynb` for the estimate and the options, and the dataset example `plausibility.python` for the check-by-check working. Both need pandas, numpy and scipy and read their inputs over HTTPS. The dataset and the WHO reference table are versioned by filename and immutable. --- ## 8. Change log | Date | Change | | --- | --- | | 2026-07-27 | First issue, against dataset v1. |