Basic Python requirements management with uv - NYCPlanning/data-engineering GitHub Wiki

Overview

Package dependencies/conflicts can be a pain in the butt, which you've probably learned if you've ever tried to continually add packages to a project/env via pip. Our preferred solution for dealing with conflicting dependencies, adding new packages to an environment, and bumping versions mainly consists of

  • declaring all of our requirements in a file named requirements.in
  • compiling them to requirements.txt via pip-tools or uv
  • installing requirements.txt in its entirely into some sort of virtual environment (uv, venv, pyenv, or conda)

This will walk through an example with just uv

Example - uv

Install uv (uv can be installed a couple ways - see full docs here)

  • unix

    curl -LsSf https://astral.sh/uv/install.sh | sh

  • Windows

    powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Create a virtual environment

uv venv --python 3.13

Activate your venv before running any more uv commands

  • unix

    source .venv/bin/activate

  • Windows

    .venv\Scripts\activate

Create your requirements.in in the root of your project

pandas
torch

Compile!

uv pip compile requirements.in --output-file requirements.txt

Result should be something like this

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.in
filelock==3.18.0
    # via torch
fsspec==2025.5.1
    # via torch
jinja2==3.1.6
    # via torch
markupsafe==3.0.2
    # via jinja2
mpmath==1.3.0
    # via sympy
networkx==3.5
    # via torch
numpy==2.3.1
    # via pandas
pandas==2.3.0
    # via -r requirements.in
python-dateutil==2.9.0.post0
    # via pandas
pytz==2025.2
    # via pandas
six==1.17.0
    # via python-dateutil
sympy==1.14.0
    # via torch
torch==2.2.2
    # via -r requirements.in
typing-extensions==4.14.0
    # via torch
tzdata==2025.2
    # via pandas

Install your packages!

uv pip install -r requirements.txt

You should be good to go! Just remember to activate the venv if you open up a new terminal.

DO check in your requirements.in and requirements.txt in git. Don't check in your virtual env! Make sure that the folder it's in is .gitignored.

[stub] - will expand later on adding new packages and updating, but it's relatively straightforward