Python Dependency Management#

Apache Superset separates Python dependency management into two distinct concerns:

1. PyPI package semantics — version ranges in pyproject.toml that give flexibility to users installing apache-superset via pip.
2. Pinned lockfiles — exact versions in requirements/*.txt files, auto-generated by uv, used for Docker images and CI/CD to guarantee reproducible builds.

A fresh pip install apache-superset may resolve different transitive dependency versions than the official Docker image, which always installs from the pinned .txt files 1.

Source of Truth: pyproject.toml#

pyproject.toml is the authoritative source for direct dependencies. It declares:

• Direct runtime deps under [project.dependencies] with range constraints (e.g., flask>=2.2.5, <3.0.0, sqlalchemy>=1.4, <2)
• Optional extras under [project.optional-dependencies] — postgres, bigquery, mysql, hive, prophet, thumbnails, and more
• Development extras grouped under the development optional dependency set

Pin overrides live in requirements/*.in files, not in pyproject.toml, keeping PyPI semantics broad.

Override Layer: requirements/*.in#

requirements/base.in supplements pyproject.toml with tighter constraints for packages known to be problematic at their full allowed range 2:

requirements/development.in installs pyproject.toml with all development and database extras: .[development,bigquery,cors,druid,gevent,gsheets,mysql,postgres,presto,prophet,trino,thumbnails].

requirements/translations.in contains only babel, used for compiling translation .mo files at image-build time.

Lockfile Generation: uv-pip-compile.sh#

All .txt lockfiles are auto-generated by scripts/uv-pip-compile.sh. It runs three uv pip compile invocations 3:

1. uv pip compile pyproject.toml requirements/base.in -o requirements/base.txt — merges direct deps with base.in overrides into a fully pinned lockfile
2. uv pip compile requirements/development.in -c requirements/base.txt -o requirements/development.txt — uses base.txt as a constraint so development deps share the same pinned base versions
3. uv pip compile requirements/translations.in -o requirements/translations.txt

To update a dependency: edit pyproject.toml or a .in file, then run ./scripts/uv-pip-compile.sh 4. Commit the resulting .txt files — Docker and CI consume them directly.

Each entry in the generated .txt files annotates its origin, e.g.:

cachetools==5.5.2
    # via google-auth

5

Any package whose only # via entry is a third-party library (not apache-superset (pyproject.toml)) is a candidate undeclared direct dependency (see below).

Docker Installation#

The Dockerfile uses multi-stage builds and installs from pre-compiled .txt files — never resolving versions at image-build time 6:

All installs go through docker/pip-install.sh, which delegates to uv pip install. uv itself is bootstrapped on the python-base layer before the virtualenv is created.

Undeclared Transitive Dependencies#

A known pitfall: packages Superset imports directly but that appear only as transitive deps — not declared in pyproject.toml. When an upstream package drops that transitive dep, a fresh pip install will break Superset at runtime.

A live example: cachetools was imported directly in Superset's AWS IAM engine spec but not declared in pyproject.toml. It arrived transitively via google-auth. When google-auth v2.53.0 dropped cachetools as a dependency, Superset 6.1.0 users hit ModuleNotFoundError: No module named 'cachetools' 7.

Rule of thumb: if Superset source has a top-level import <package>, that package must be in pyproject.toml. The # via annotation in the .txt lockfiles is the fastest audit tool.

Python Version Compatibility#

CI runs the test suite on both the current stable Python version and a "next" canary version (currently Python 3.13) 8. The canary runs as a dedicated nightly workflow (Python-Nightly-Canary) rather than on every push, to catch cross-version breakage early without paying for a second full Python version on every merge.

Known cross-version pitfalls that have required dependency changes:

• zope-interface<7.2 fails on Python 3.13 — the interface metaclass treated __firstlineno__ (a new implicit attribute in Python 3.13) as an interface member, crashing gevent on import. Fixed by pinning zope.interface>=7.2 in requirements/development.in 8.

License Compliance#

pyproject.toml configures liccheck to audit requirements/base.txt against an allowlist of approved licenses. Two GPL-licensed packages (paramiko, pyxlsb) are currently flagged with a # TODO REMOVE THESE DEPS FROM CODEBASE comment.