cassionData Analysis

Back to the lessonLesson 1 of 8An environment that will still run

A Python you can reinstall without internet

The same deck as the downloads, rendered as a page. Start the slideshow to present it full screen — arrow keys or a click advance one slide, Escape leaves.

Slides · PDFSlides · PowerPoint

  1. Slide 1 / 19

    What this lesson covers

    • The failure this lesson prevents
    • Do not work in the system Python
    • What a virtual environment is
    • uv, and why it matters more here than elsewhere
    • Installing where there is no internet
    • Pinning the Python version too
    • Verifying the environment before trusting it
    • What to hand to a colleague
    • What comes next
    Speaker notes
    Why the system Python is the wrong place to work, what a virtual environment actually is, and how to carry a working install to a field laptop that has never seen a package index.
  2. Slide 2 / 19

    The failure this lesson prevents

    • An analysis runs on your laptop.
    Speaker notes
    An analysis runs on your laptop. Three months later the same script is run by someone else, on a different machine, and it either fails to start or — much worse — produces a different number. That is not usually a bug in the analysis. It is a bug in the environment: a different pandas version whose default changed, a package installed for one project that broke another, a system Python the operating system upgraded underneath you. This lesson is about making the environment a thing you declare, rather than a thing that accumulates.
  3. Slide 3 / 19

    Do not work in the system Python — Example

    error: externally-managed-environment
    
    x This environment is externally managed
    +-> To install Python packages system-wide, try apt install
        python3-xyz, where xyz is the package you are trying to
        install.
    Speaker notes
    macOS and most Linux distributions ship a Python that the operating system uses for its own tooling. On a shared field laptop, someone has usually installed a second one from a website, and a third arrived with an office suite. Installing packages into any of them is a bad idea for a specific reason: the operating system depends on the versions it shipped. Upgrading pandas to satisfy your script can break a system utility, and there is nothing to warn you. Newer Python builds now refuse outright:
  4. Slide 4 / 19

    Do not work in the system Python

    • That error is correct and should not be worked around — There is a flag that overrides it
    Speaker notes
    That error is correct and should not be worked around. There is a flag that overrides it. Using it is how a laptop ends up needing to be reimaged.
  5. Slide 5 / 19

    What a virtual environment is — Shell

    python3 -m venv .venv
    source .venv/bin/activate       # Windows: .venv\Scripts\activate
    python -m pip install pandas
    Speaker notes
    A virtual environment is a directory holding its own copy of the interpreter's package directory. Activating it changes where python and pip look. That is the whole idea.
  6. Slide 6 / 19

    What a virtual environment is

    • Per project, not per machine. Two analyses needing different pandas versions coexist without either knowing about…
    • Disposable. If an environment gets into a bad state, delete the directory and rebuild it. Nothing of value lives…
    Speaker notes
    Two consequences worth internalising:
  7. Slide 7 / 19

    uv, and why it matters more here than elsewhere — Shell

    # Install uv itself, once, on a machine that does have internet.
    curl -LsSf https://astral.sh/uv/install.sh | sh
    
    uv init muac-analysis
    cd muac-analysis
    uv add pandas openpyxl
    uv run python -c "import pandas; print(pandas.__version__)"
    Speaker notes
    pip resolves and downloads packages one at a time from the network. On a connection that drops, a half-finished install leaves an environment that is neither the old one nor the new one. uv is a faster resolver that produces a lockfile: an exact list of every package and version, including the ones your packages depend on.
  8. Slide 8 / 19

    uv, and why it matters more here than elsewhere — Shell

    uv sync                        # reads uv.lock, installs exactly those versions
    Speaker notes
    uv add writes two files. pyproject.toml records what you asked for — pandas — and uv.lock records what you got, down to the exact version of every transitive dependency. Commit both. The first is your intent; the second is what makes next quarter's install identical to this one. On the field laptop: conda is the reasonable alternative and is common in this sector, particularly where a geospatial stack is involved — GDAL is far easier to install through conda than through pip. If your organisation already standardises on conda, use conda; the principle is identical and only the commands change.
  9. Slide 9 / 19

    Installing where there is no internet — Shell

    mkdir wheels
    uv pip download pandas openpyxl --dest wheels
    Speaker notes
    This is the part most tutorials skip, and it is the part that matters on a deployment. Download the packages once, on a connected machine, targeting the platform the field laptop runs:
  10. Slide 10 / 19

    Installing where there is no internet — Shell

    uv venv
    uv pip install --no-index --find-links wheels pandas openpyxl
    Speaker notes
    Copy the wheels directory onto a USB stick along with your project, then on the offline machine:
  11. Slide 11 / 19

    Installing where there is no internet

    Wheels are platform-specific. A wheel downloaded on macOS will not install on a Windows laptop, and one built for Python 3.12 will not install into 3.11. Download on a machine matching the target, or pass --python-platform and --python-version explicitly.
    Speaker notes
    --no-index tells the installer not to contact the package index at all, and --find-links points it at the directory instead. The install is then entirely local.
  12. Slide 12 / 19

    Pinning the Python version too — Example

    # pyproject.toml
    [project]
    name = "muac-analysis"
    requires-python = ">=3.11"
    dependencies = ["pandas>=2.2", "openpyxl>=3.1"]
    Speaker notes
    The packages are pinned; the interpreter should be as well. A script written against 3.12 and run on 3.9 fails on syntax that did not exist yet.
  13. Slide 13 / 19

    Pinning the Python version too — Shell

    uv python install 3.12
    uv python pin 3.12
  14. Slide 14 / 19

    Verifying the environment before trusting it — In Python

    import sys
    import pandas as pd
    
    print(sys.executable)          # should be inside .venv, not /usr/bin
    print(sys.version)
    print(pd.__version__)
    Speaker notes
    Before running an analysis on a machine you did not set up, check that you are where you think you are:
  15. Slide 15 / 19

    Verifying the environment before trusting it — In Python

    import sys
    
    assert sys.version_info >= (3, 11), f"needs Python 3.11+, running {sys.version}"
    Speaker notes
    If sys.executable is /usr/bin/python3, the environment is not activated and you are about to install into the system Python. That single line has saved more laptops than any amount of documentation. For an analysis whose numbers matter, assert it:
  16. Slide 16 / 19

    What to hand to a colleague

    CommitDo not commit
    pyproject.toml.venv/
    uv.lockwheels/
    README.md with the two commands__pycache__/
    Speaker notes
    Everything needed to rebuild the environment, and nothing that was built:
  17. Slide 17 / 19

    What to hand to a colleague — Example

    uv sync
    uv run python scripts/indicator_table.py --input data/raw/muac.csv
    Speaker notes
    A .gitignore covering the right column is part of the project template in the next lesson. The README needs two lines, not two pages:
  18. Slide 18 / 19

    What comes next

    • The environment is reproducible.
    Speaker notes
    The environment is reproducible. The next lesson is about the project around it — where the raw data goes, where outputs go, and why a path written as C:\Users\marie\Desktop\data.csv guarantees the script runs on exactly one machine.
  19. Slide 19 / 19

    Where this goes next

    Read the full lesson, with runnable code Back to the lesson