Backend Bundle Failures From Optional Dependencies#

Lead Section#

Backend bundle failures from optional dependencies occur when JavaScript bundlers attempt to include transitive or optional dependencies—particularly native Node.js addons—that are not required for the application's runtime execution. This class of build failures is especially common in containerized environments where bundlers traverse dependency trees and encounter platform-specific native modules from features the application doesn't actually use.

A canonical example involves the dependency chain from Docker client libraries: an application using dockerode may transitively depend on docker-modem → ssh2 → cpu-features, where cpu-features is a native addon for CPU capability detection in SSH encryption. When a bundler like Bun or esbuild attempts to bundle this chain, it tries to resolve the native module cpufeatures.node even though the application never uses SSH-based Docker connections. The build fails with resolution errors like "Could not resolve: ../build/Release/cpufeatures.node" because the native binary doesn't exist in the build environment.

The primary resolution strategy is dependency externalization: configuring the bundler to exclude specific dependencies from the bundle and resolve them at runtime from node_modules instead. This approach allows applications to ship with the full dependency tree in their deployment artifacts while avoiding bundler failures on unused optional features.

Problem Description#

Root Cause: Eager Loading of Optional Features#

Modern JavaScript packages often include optional features that are conditionally used based on runtime configuration. However, many packages implement these features through eager loading patterns where the module system loads all feature code at import time, even if the features are never invoked.

For example, docker-modem (the HTTP/stream transport layer for Docker clients) supports multiple connection protocols including Unix sockets, HTTP, and SSH. The library structure requires all transport modules at the top level, so importing docker-modem causes the module system to traverse into SSH transport code, which in turn loads the ssh2 package and its native dependency cpu-features.

This becomes problematic during bundling because:

1. Bundlers perform static analysis of the entire dependency graph, traversing all require() and import statements
2. Native modules require compilation artifacts (.node files) that may not exist in the build environment
3. Platform-specific binaries may not match the build platform even if they exist in node_modules

Symptom: Build Failures with Cryptic Error Messages#

Build failures from optional native dependencies often manifest with minimal diagnostic information. Tool exit codes may be the only visible symptom, with the actual resolution error buried in build logs. Common error patterns include:

• "Could not resolve: ../build/Release/[module].node"
• "Cannot find module '../../build/Release/[module].node'"
• Exit code 1 or 2 with no additional context in high-level build orchestrators

These errors are particularly challenging to debug because:

• The failing module is often several levels deep in the dependency tree
• The module isn't imported anywhere in application code
• The error appears non-deterministically based on bundler caching or dependency installation order

Dependency Chain Analysis#

The dockerode → cpu-features Example#

The dockerode library provides a Node.js Docker client. While most applications use it for standard Docker socket connections, the library's architecture creates a problematic transitive dependency chain:

Application Code
└── dockerode (Docker client)
    └── docker-modem (Transport layer)
        └── ssh.js (SSH transport module - loaded eagerly)
            └── ssh2 (SSH protocol implementation)
                └── cpu-features (Native addon for CPU capability detection)

Key characteristics of this chain:

1. docker-modem eagerly loads SSH support: The main modem.js file requires ./ssh.js regardless of whether SSH connections are used
2. ssh2 eagerly loads cpu-features: The SSH library loads the native addon at module initialization to detect CPU features for cryptographic optimization
3. cpu-features requires native compilation: The package includes C++ code that must be compiled to a .node binary for the target platform

Why Optional Dependencies Appear in Bundles#

The npm/Node.js ecosystem distinguishes between several dependency types:

• dependencies: Required for the package to function
• devDependencies: Only needed during development
• peerDependencies: Must be provided by the consuming application
• optionalDependencies: Provide enhanced functionality if available, but the package works without them

However, optionalDependencies are still installed by default with npm install or bun install. Packages like ssh2 list cpu-features as optional because SSH connections work without CPU feature detection (falling back to generic implementations), but the module is loaded unconditionally if present.

Bundlers see these installed optional dependencies as part of the available module graph and attempt to bundle them alongside required dependencies.

Resolution Strategies#

Strategy 1: Externalize Dependencies#

Externalizing a dependency instructs the bundler to exclude it from the output bundle and instead resolve it at runtime from node_modules. This is the most common solution for native modules and large dependency trees.

Implementation in different bundlers:

Vite/Rollup Configuration:

// vite.config.ts
export default defineConfig({
  build: {
    rollupOptions: {
      external: ["dockerode", "bun"],
    },
  },
});

This approach is used in zerobyte's vite.config.ts, which externalizes the bun runtime to prevent bundling runtime-specific APIs.

Bun Bundler Configuration:

// build.ts
await Bun.build({
  entrypoints: ['./src/index.ts'],
  outdir: './dist',
  external: ['dockerode', 'ssh2', 'cpu-features'],
});

Webpack Configuration:

// webpack.config.js
module.exports = {
  externals: {
    dockerode: 'commonjs dockerode',
  },
};

Tradeoffs:

• ✅ Pros: Simple configuration, preserves all functionality, no code changes required
• ❌ Cons: Requires shipping full node_modules in deployment artifacts, larger Docker images

Strategy 2: Prevent Installation with --ignore-scripts#

For Docker builds and CI environments, preventing native module compilation entirely can avoid build failures:

# Dockerfile
RUN bun install --frozen-lockfile --ignore-scripts

This pattern is demonstrated in zerobyte's Dockerfile where --ignore-scripts prevents optional dependency installation scripts from executing. This flag:

• Skips postinstall, install, and other lifecycle scripts
• Prevents native module compilation (node-gyp, etc.)
• Still installs the JavaScript portions of packages

Tradeoffs:

• ✅ Pros: Faster builds, avoids native toolchain requirements in build containers
• ❌ Cons: Breaks packages that legitimately require compilation, may cause runtime failures if native modules are actually needed

Strategy 3: Remove Unused Dependencies#

When features enabled by optional dependencies are never used, the most robust solution is removing the dependency entirely:

npm uninstall dockerode
# or remove from package.json

Zerobyte PR #137 demonstrates this approach, where Docker volume plugin functionality and the dockerode dependency were removed entirely when the feature was no longer needed.

Tradeoffs:

• ✅ Pros: Eliminates the problem permanently, reduces bundle size and attack surface
• ❌ Cons: Requires refactoring application code, may not be feasible if the feature is needed

Strategy 4: Use Runtime-Native Alternatives#

Modern runtimes like Bun and Deno provide native implementations of common functionality, eliminating the need for native Node.js addons:

Example: Database Connections

Instead of using better-sqlite3 (a native addon):

// Node.js with native addon
import Database from 'better-sqlite3';
const db = new Database('data.db');

Use Bun's native SQLite:

// Bun native
import { Database } from 'bun:sqlite';
const db = new Database('data.db');

Zerobyte uses this pattern extensively, with bun:sqlite instead of native Node.js database drivers.

Tradeoffs:

• ✅ Pros: No native compilation, better performance, simplified bundling
• ❌ Cons: Runtime lock-in, may not have feature parity with npm packages

Strategy 5: Pin Bundler Versions#

Bundler behavior can change between versions, sometimes introducing regressions in native module handling. Dependency version overrides can ensure consistent behavior:

{
  "overrides": {
    "esbuild": "^0.27.2"
  }
}

Zerobyte PR #367 demonstrates synchronizing overrides to prevent "runtime errors, build failures, or unexpected behavior" from bundler version mismatches.

Best Practices#

1. Understand Your Dependency Tree#

Use tooling to visualize transitive dependencies and identify problematic chains:

# npm
npm ls dockerode
npm ls cpu-features

# bun
bun pm ls dockerode

# yarn
yarn why cpu-features

This reveals the exact path from your application code to the problematic native module.

2. Externalize Cautiously#

Don't externalize everything to avoid thinking about bundling. Each externalized dependency:

• Must be installed in the deployment environment
• Increases node_modules size in Docker images
• May have its own transitive dependencies with vulnerabilities

Guideline: Externalize at the highest level in the dependency chain that resolves the build failure. If dockerode → docker-modem → ssh2 → cpu-features is failing, externalize dockerode rather than each individual package.

3. Test Bundled vs. Unbundled Builds#

Bundled applications have different runtime behavior than unbundled ones:

• Module resolution paths change
• Dynamic require() may fail
• File system assumptions may break

Always test with NODE_ENV=production using your built artifacts, not ts-node or tsx development servers.

4. Document Why Dependencies Are Externalized#

Future maintainers need to understand why externalization was necessary:

// vite.config.ts
export default defineConfig({
  build: {
    rollupOptions: {
      // Externalize dockerode to avoid bundling cpu-features native addon
      // from docker-modem -> ssh2 -> cpu-features chain
      // See: https://github.com/org/repo/issues/123
      external: ["dockerode"],
    },
  },
});

5. Consider Alternative Architectures#

If optional dependencies consistently cause build issues, consider whether the architecture can change:

• Use external system binaries instead of npm packages (restic, rclone, etc. as demonstrated in zerobyte's Dockerfile)
• Split Docker client logic into a separate microservice that doesn't need bundling
• Use HTTP APIs instead of client libraries (Docker's HTTP API vs. dockerode)

Relevant Code Files#

Related Topics#

• Native Node.js Addons: C++ modules compiled for specific platforms that require node-gyp or similar build toolchains
• Dependency Tree Shaking: Bundler optimization that removes unused code, but struggles with dynamic requires
• Optional Dependencies: npm package.json field for dependencies that provide enhanced functionality if available
• Docker Layer Caching: Build optimization strategy that can mask or exacerbate bundling issues depending on layer ordering
• Monorepo Build Orchestration: Tools like Turborepo and Nx that can hide build failures in summarized output