cassionData Analysis

Back to the lessonLesson 5 of 8The report around it

Findings, limitations, recommendation — in that order and separable

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 19

    What this lesson covers

    • Who reads what
    • The order the platform's reports use
    • The recommendation paragraph
    • Limitations that are not apologies
    • The three sentences every report needs
    • Length, and what to cut
    • Report it whole
    • What comes next
    Speaker notes
    Somebody will read only the recommendation, somebody else only the limitations, and a third person will forward one paragraph with no context. Write the report so each of those readings is survivable, because all three happen to every report.

  2. Slide 2 / 19

    Who reads what

    ReaderReadsNeeds
    The decision-makerThe recommendationWhat to do, and what it costs
    The technical reviewerThe limitations and the annexWhether to believe it
    The donorThe summaryWhether the money did something
    Whoever forwards itOne paragraphThat paragraph to be true alone
    Speaker notes
    A programme report has four readers and none of them reads all of it.

  3. Slide 3 / 19

    Who reads what

    • Write for the fourth — A report whose paragraphs survive being separated is a report that survives its own circulation,…
    Speaker notes
    Write for the fourth. A report whose paragraphs survive being separated is a report that survives its own circulation, and every other reader is served by the same discipline.

  4. Slide 4 / 19

    The order the platform's reports use — Example

    ## Recommendation
## What changed in the measurement
## The ranking
## Why the list can be defended
## Water quality, reported separately
## What this does not establish
## Recommended next steps
    Speaker notes
    Every project report here has the same shape, and it is not the academic one.

  5. Slide 5 / 19

    The order the platform's reports use

    • The recommendation is first — Not the methodology, not the background — the thing you are asking someone to do
    • "What this does not establish" is a numbered section, not a footnote — It is where the technical reviewer goes first,…
    • The methodology is not in the report at all — It is in the indicator definitions document, linked, because a reader who…
    Speaker notes
    The recommendation is first. Not the methodology, not the background — the thing you are asking someone to do. A reader who stops after one section should have the answer. "What this does not establish" is a numbered section, not a footnote. It is where the technical reviewer goes first, and burying it signals that you would rather they did not. The methodology is not in the report at all. It is in the indicator definitions document, linked, because a reader who wants it wants all of it and a reader who does not wants none.

  6. Slide 6 / 19

    The recommendation paragraph

    • What to do — A verb and an object
    • Why these and not others — One sentence of criterion, not a summary of the analysis
    • What it costs, or what it displaces — A recommendation with no cost is a wish
    • What would change it — The condition under which this advice stops being the advice — which is what makes it a…
    Speaker notes
    Four things, and it fails without any of them. What to do. A verb and an object. "Rehabilitate the water points in these six communities." Why these and not others. One sentence of criterion, not a summary of the analysis. What it costs, or what it displaces. A recommendation with no cost is a wish. What would change it. The condition under which this advice stops being the advice — which is what makes it a recommendation rather than an instruction.

  7. Slide 7 / 19

    The recommendation paragraph — Example

    Rehabilitate water points in the six communities listed in section 3.

They are the six with the highest combined need score: households above the
thirty-minute collection threshold, households below the Sphere quantity
minimum, and households on an unimproved source. The score is defined in
docs/indicator-definitions.md section 4.

This is the full rehabilitation budget for the cycle. Three of the six were
not in the previous list, which was ranked on improved-source counts.

If the second survey round shows the three new communities improving without
intervention, the list should be recomputed before the following cycle.

  8. Slide 8 / 19

    The recommendation paragraph

    • The third paragraph is the one that gets the recommendation accepted or rejected — because it says what changes rather…
    Speaker notes
    The third paragraph is the one that gets the recommendation accepted or rejected, because it says what changes rather than what is true.

  9. Slide 9 / 19

    Limitations that are not apologies

    • Not this — "The data have some limitations and results should be interpreted with caution." This says nothing and reads…
    • This — "Water quality was tested in 33.4% of households
    Speaker notes
    There is a register that undermines a report and a register that strengthens it, and the difference is whether the limitation is quantified. Not this: "The data have some limitations and results should be interpreted with caution." This says nothing and reads as a disclaimer. This: "Water quality was tested in 33.4% of households. The quality figures are reported on that subsample and are not comparable with the access figures, which use all 2,403."

  10. Slide 10 / 19

    Limitations that are not apologies

    WeakStrong
    "Small sample in some areas""Three communes have fewer than 200 screenings; their intervals are twice as wide"
    "Data quality issues""One team's mean sits beyond the SMART concern threshold; its clusters are excluded from the prevalence"
    "Results are indicative""This survey cannot distinguish one commune from its neighbour; it can distinguish these three from the district median"

  11. Slide 11 / 19

    Limitations that are not apologies

    • A quantified limitation is a finding — It tells a reader what the analysis can and cannot support, which is more useful…
    Speaker notes
    A quantified limitation is a finding. It tells a reader what the analysis can and cannot support, which is more useful than a general warning and much harder to ignore.

  12. Slide 12 / 19

    The three sentences every report needs

    • The estimate with its interval, in programme units — "19.4 percentage points, or 38 cases that did not reach a service."
    • The smallest effect the design could detect — "This evaluation could not have detected anything below 4.9 points."
    • What the analysis cannot rule out — "Effects between zero and three points — the range the programme is arguing about —…
    • Most reports have the first and not the second or third — and the absence of the second is what lets a null result be…
    Speaker notes
    They come from the statistics and impact evaluation courses and they belong in every narrative report this audience writes. The estimate with its interval, in programme units. "19.4 percentage points, or 38 cases that did not reach a service." The smallest effect the design could detect. "This evaluation could not have detected anything below 4.9 points." What the analysis cannot rule out. "Effects between zero and three points — the range the programme is arguing about — are not excluded." Most reports have the first and not the second or third, and the absence of the second is what lets a null result be read as a refutation.

  13. Slide 13 / 19

    Length, and what to cut

    • Four to six pages for a programme report — and the discipline is in what is not there
    Speaker notes
    Four to six pages for a programme report, and the discipline is in what is not there.

  14. Slide 14 / 19

    Length, and what to cut

    CutBecause
    The methodology sectionIt is the definitions document, linked
    The literature reviewNobody reading this is deciding whether the field is real
    Every chart that does not carry a findingA chart is a claim; a decorative one is a claim about nothing
    The background on the crisisThe audience lives there
    Anything the annex holdsThat is what an annex is

  15. Slide 15 / 19

    Length, and what to cut

    • The one thing never to cut is the "does not establish" section — It is the first thing an editor reaches for and the…
    Speaker notes
    The one thing never to cut is the "does not establish" section. It is the first thing an editor reaches for and the last thing that should go.

  16. Slide 16 / 19

    Report it whole — Example

    Report structure

  Recommendation (1 page), what changed in the measurement, the ranked list,
  why the list can be defended, water quality reported separately, what this
  does not establish, next steps.

  Methodology is not in the report. Indicator definitions, with numerator,
  denominator and standard for each of nine indicators, are in
  docs/indicator-definitions.md and linked from the recommendation.

  Each section is written to be readable alone, because the recommendation
  will be forwarded without the rest of it.

  The report states the interval on every estimate and names one thing the
  analysis cannot establish for each of its three findings.

  17. Slide 17 / 19

    Report it whole

    • "Written to be readable alone" is a testable claim — Send one section to somebody and ask what it says
    Speaker notes
    "Written to be readable alone" is a testable claim. Send one section to somebody and ask what it says.

  18. Slide 18 / 19

    What comes next

    • A report that states its limitations invites a reviewer to check them.
    Speaker notes
    A report that states its limitations invites a reviewer to check them. The next lesson is about the annex that lets them — and about why an annex full of only successful analyses is the one nobody believes.

  19. Slide 19 / 19

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson