Poetry's Universal Dependency Resolution#

Poetry takes a unique approach to dependency resolution that sets it apart from simpler package installers like pip. This document explains how Poetry's universal dependency resolution works, the complexities involved, and the assumptions and quirks you should be aware of.

What is Universal Dependency Resolution?#

When you run poetry lock or poetry install, Poetry resolves dependencies for all the Python versions you've declared in your project, not just the Python version you're currently using. This is called universal dependency resolution.

For example, if your pyproject.toml declares:

[tool.poetry.dependencies]
python = "^3.8"

Poetry will ensure that all dependencies work for Python 3.8, 3.9, 3.10, 3.11, 3.12, and beyond—not just the Python version running on your machine right now. This is fundamentally different from pip, which only resolves dependencies for your current environment.

The goal is to create a cross-platform lock file that works across all target environments. When a team member with Python 3.9 on Windows and another with Python 3.12 on Linux both run poetry install, they should get consistent, compatible dependencies.

Why Universal Resolution Matters#

Universal resolution prevents a common problem: your project working on your machine but breaking for teammates or users with different Python versions. Poetry validates that all dependencies are resolvable across your entire Python version range, catching incompatibilities before they cause runtime failures.

However, this approach comes with tradeoffs. Resolution can be slower and more complex than single-environment resolution, and you need to be precise about your Python version constraints.

The PubGrub Algorithm#

Poetry uses the PubGrub version solving algorithm, originally developed by the Dart team. Unlike simple backtracking algorithms that try versions one by one, PubGrub uses a sophisticated approach called conflict-driven clause learning.

How PubGrub Works#

The algorithm operates in three phases:

1. Building incompatibilities: Recording sets of package version constraints that cannot be satisfied together. For example, if package A version 1.0 requires package B < 2.0, but package C requires B >= 2.0, this is an incompatibility.

2. Unit propagation: Using these incompatibilities to automatically derive new assignments. If you know that package A must be installed and A requires B < 2.0, you can immediately constrain B's version.

3. Conflict resolution with backtracking: When conflicts arise, the algorithm analyzes them to learn new incompatibilities and performs intelligent "backjumping" to earlier decision points, skipping large portions of the search space.

This approach is much more efficient than naive backtracking because it learns from conflicts. If trying package X version 2.0 causes a conflict deep in the dependency tree, PubGrub records why and won't waste time trying X 2.0 again in other branches.

Key Data Structures#

Poetry's implementation uses several core data structures:

• Incompatibility: A set of package version constraints that cannot be satisfied simultaneously
• Term: A positive or negative constraint on package versions
• PartialSolution: Tracks the solver's current assignments and decisions

How Universal Locking Works#

The real complexity of Poetry's resolver lies in how it handles multiple Python versions and platforms simultaneously. This is where environment markers come into play.

Python Constraint Handling#

The Provider class maintains a _python_constraint representing your project's Python version range. During resolution:

• Dependencies are filtered based on Python compatibility
• Packages with incompatible Python requirements generate incompatibilities that guide the solver away from those versions

Environment Markers: The Complexity#

Environment markers are conditions like python_version >= "3.8" or sys_platform == "win32" that specify when a dependency is needed. They're defined in PEP 508 and supported by Poetry through dependency specification syntax.

Markers make resolution complex because:

• A single dependency might need different versions on different platforms
• Multiple overlapping markers can apply to the same package
• Markers must be propagated through the entire dependency tree
• The final lock file must contain mutually exclusive markers to avoid ambiguity

Marker Propagation#

Each dependency carries a transitive_marker that tracks when it's needed. After resolution completes, the calculate_markers() function computes final markers by propagating them from the root of the dependency tree.

For example, if your project depends on requests only on Windows (sys_platform == "win32"), and requests depends on urllib3, then urllib3 also gets the marker sys_platform == "win32" because it's only needed when requests is needed.

Overlapping Marker Transformation#

Starting with Poetry 1.6.0, Poetry fully supports duplicate dependencies with overlapping markers. This is handled by _resolve_overlapping_markers(), which uses boolean algebra to transform overlapping markers into mutually exclusive ones.

How it works:

Given dependencies with overlapping markers like:

pypiwin32 (220); sys_platform == "win32" and python_version >= "3.6"
pypiwin32 (219); sys_platform == "win32" and python_version < "3.6"

The algorithm generates all 2^n combinations of markers and their inverses:

• (marker1) AND (marker2) — both conditions true
• (marker1) AND (NOT marker2) — only first condition true
• (NOT marker1) AND (marker2) — only second condition true
• (NOT marker1) AND (NOT marker2) — neither condition true

It then filters to keep only relevant intersections (non-empty, within Python constraints) and assigns packages to each exclusive region. This ensures that for any given environment, exactly one version is selected.

Marker Simplification#

After markers are computed, they go through multiple levels of simplification to keep the lock file readable:

Single marker merging: Markers with the same attribute are merged by intersecting or unioning their constraints. For example:

• python_version >= "3.8" and python_version < "3.9" → python_version == "3.8"

MultiMarker and MarkerUnion simplification: The system detects when one marker subsumes another and finds common markers to simplify unique ones separately:

• python_version >= "3.8" and python_version < "3.10" or python_version >= "3.10" and python_version < "3.12"
• Simplifies to: python_version >= "3.8" and python_version < "3.12"

CNF/DNF normalization: Markers are converted to Conjunctive or Disjunctive Normal Form and the simplest representation is chosen based on complexity.

Python Version vs. Platform Markers#

Poetry handles different types of markers differently:

Python version markers (python_version, python_full_version):

• Use constraint-based merging with sophisticated range operations
• Can be simplified aggressively (e.g., converting ranges to equality)
• Special simplifications can eliminate redundant conditions:
  • python_version >= "3.8" and sys_platform == "linux" or python_version > "3.6" → python_version > "3.6"

Platform markers (sys_platform, platform_system, platform_machine):

• Treated as generic string constraints without version logic
• Use simple equality/inequality comparisons
• Merged using union/multi constraints
• Example: sys_platform == "linux" or sys_platform == "win32" stays as-is

Extra markers: The extra marker is uniquely special because multiple extras can be active simultaneously. Unlike other markers, extra == "a" and extra == "b" is not considered empty—both extras can be active at once.

Compatibility Mode with Overrides#

Sometimes a package has conflicting dependencies for different Python versions. For example:

package-foo requires bar >= 2.0 on Python >= 3.6
package-foo requires bar < 2.0 on Python < 3.6

When Poetry detects such conflicts, it:

1. Raises an OverrideNeededError
2. Enters compatibility mode via _solve_in_compatibility_mode()
3. Solves separately with each override (one for each marker condition)
4. Merges the results using merge_override_packages(), attaching appropriate markers to each package

This allows the lock file to contain different package versions for different Python versions, all resolved in a coordinated way.

The Resolution Process#

Here's how a complete resolution flows from start to finish:

Phase 1: Initialization#

The Solver class creates a Provider and initializes state. The Provider acts as the interface to package repositories and handles metadata gathering.

Phase 2: Version Solving#

VersionSolver.solve() enters the main PubGrub loop:

1. Creates the root incompatibility
2. Alternates between:
   • _propagate(): Performs unit propagation using known incompatibilities
   • _choose_package_version(): Selects the next package version to try
3. On conflicts, calls _resolve_conflict() to learn and backjump

If an override is needed for compatibility mode, the solver catches the exception and re-solves with the necessary overrides.

Phase 3: Post-Processing#

After solving, _aggregate_solved_packages() performs several critical steps:

1. Performs a depth-first search on the dependency graph to compute tree depth
2. Calculates final markers for each package by propagating from the root
3. Simplifies markers to remove redundant Python version constraints

Phase 4: Transaction to Operations#

The solver returns a Transaction containing the resolved packages with their markers. Finally, Transaction.calculate_operations() converts the solution into concrete install, update, and uninstall operations.

Package Selection Heuristics#

When choosing which package version to try next, Poetry's decision-making prioritizes packages in this order:

1. Direct origin dependencies (VCS, path, URL) — resolved first
2. Packages with no choice (only one version available)
3. Packages requested with --use-latest
4. Locked packages (from existing lockfile)
5. Default: Packages with more versions available, prioritizing those with dependencies that have upper bounds

The last heuristic differs from the original PubGrub proposal, which prefers packages with fewer versions. Poetry found that preferring packages with more versions reduces resolution time for Python packages, likely because it's more likely to find a compatible version quickly.

Metadata Gathering#

Before resolving, Poetry needs to know each package's dependencies. It uses a layered approach:

1. PyPI packages: Fetches from https://pypi.org/pypi/{package}/json
2. Private repositories: Uses the simple endpoint
3. Fallback strategies: Infers from filename if metadata unavailable, then downloads
4. Wheels: Extracts static metadata directly
5. Source distributions: Unpacks and collects static metadata; falls back to PEP 517 metadata build if incomplete
6. Last resort: Parses setup.py

The Lazy Wheel Setting#

The solver.lazy-wheel configuration (default true) uses HTTP range requests to download only metadata files instead of full wheels. This speeds up resolution, especially with slow networks.

However, when enabled, Poetry may miss platform-specific dependencies if the PyPI JSON API metadata is incomplete. This is not a Poetry bug—it's a package metadata problem. Package maintainers should publish wheels with consistent metadata using PEP 508 markers.

If you encounter missing platform-specific dependencies, try:

poetry config solver.lazy-wheel false

Critical Assumptions#

All Dependencies Must Work for All Python Versions#

This is the most important assumption: all defined dependencies must be valid for all defined Python versions of your project.

If your project declares python = ">=3.7" but a dependency requires python >= "3.7.1", Poetry will fail because your declared range technically includes python == 3.7.0, which the dependency doesn't support.

The fix is simple—tighten your Python constraint to match:

[tool.poetry.dependencies]
python = "^3.7.1"

As a maintainer stated: "Ignoring the upper boundary of supported python version isn't an option. This would be a wild guess by the resolver... So this behavior by poetry will never change."

This strictness is intentional. It prevents your project from claiming support for Python versions where it would actually break at runtime.

Known Quirks and Edge Cases#

1. Dependency Graph Splitting#

Poetry prefers not to split the dependency graph, but adding a new dependency can force a split. When this happens, it can cause version changes in seemingly unrelated packages.

For example, adding cloud-sql-python-connector might cause pytest-asyncio to jump from version 1.2.0 to 1.3.0, even though these packages appear unrelated. This occurs because the split creates different resolution branches with different constraints.

This is expected behavior and an implementation detail of the resolution algorithm, not a bug.

2. Lock File Structure: python-versions vs. markers#

The lock file contains two distinct fields that are often confused:

• python-versions: Comes from the package metadata itself (the package author's declared constraint)
• markers: Generated by Poetry during resolution (the actual conditions for using this lock file entry)

When Poetry resolves multiple versions of the same package, it generates mutually exclusive markers between entries to ensure only one version is selected for any given environment.

3. The installer.re-resolve Bug#

There's a known bug with installer.re-resolve = true (the default in Poetry 2.0+): Poetry may incorrectly install packages that don't match the current Python version's markers. For instance, a package marked for Python >= 3.14 might be installed on Python 3.10.

Workaround:

poetry config installer.re-resolve false

When set to false, Poetry evaluates locked markers to decide what to install rather than re-resolving, which avoids this issue.

4. Platform-Specific Dependencies and Metadata Issues#

When PyPI's JSON API metadata is incomplete or doesn't include platform-specific dependencies with proper PEP 508 markers, Poetry may miss these dependencies. This manifests as packages that work fine on one platform but have missing dependencies on another.

This is not a Poetry bug but a package metadata problem. The proper solution requires package maintainers to fix their metadata. As a workaround, try disabling lazy wheel mode:

poetry config solver.lazy-wheel false

5. pip install and Lock Files#

When a Poetry-managed package is installed via pip install, pip calls the build backend per PEP 517, which uses version ranges from pyproject.toml—poetry.lock is not consulted.

This is intentional behavior. Lock files are meant for applications (things you deploy), not libraries (things others depend on). When someone installs your library, they get the flexibility of version ranges, not the strict versions from your lock file.

Performance Considerations#

Dependency resolution performance has improved across Poetry versions:

• Poetry 1.1.0 improved the algorithm's accuracy for cases with multiple constraint dependencies, though this sometimes came at the cost of speed
• Poetry 1.6.0 added performance improvements for specific edge cases, particularly addressing issues with packages like boto3/botocore and urllib3
• Poetry 1.8.0 introduced lazy wheel loading, which uses HTTP range requests to speed up metadata gathering with slow networks

If resolution is slow for your project, consider:

• Constraining packages more narrowly—broad version ranges force Poetry to examine more possibilities
• Using the --use-latest flag sparingly—it disrupts the optimization from locked packages
• Checking if metadata issues are causing excessive downloads

Summary#

Poetry's universal dependency resolution is powerful but complex. It ensures your project works across all declared Python versions and platforms by:

1. Using the PubGrub algorithm for efficient conflict-driven solving
2. Propagating and simplifying environment markers through the dependency tree
3. Transforming overlapping markers into mutually exclusive ones using boolean algebra
4. Validating that all dependencies work across your entire Python version range

The key assumptions to remember:

• All dependencies must support all your declared Python versions
• Lock files are universal—they work across platforms and Python versions
• Markers in the lock file are generated by Poetry and may differ from package metadata

Understanding these mechanisms helps you write better pyproject.toml files, debug resolution issues, and make informed decisions about version constraints and markers.