Crate Packaging & Path Portability#
When OpenDAL's workspace was split into publishable sub-crates (RFC-6828), file-path assumptions that work in a git checkout began to break in crates.io tarballs. The core issue: include_str! paths that traverse above a sub-crate's own directory resolve correctly in the monorepo but point to non-existent locations in a published package. This issue has been resolved through a combination of path correction and symlink addition as of PR #8223.
The Breaking Pattern#
core/core/src/docs/mod.rs embeds the project changelog. Before the fix, it used:
#[doc = include_str!("../../../CHANGELOG.md")]
pub mod changelog {}
In the repository, ../../../ from core/core/src/docs/ resolved to the repo root, where CHANGELOG.md lives. In a crates.io tarball, however, only the core/core/ subtree is packaged β there is no parent directory, and the path resolved to nothing. This has been corrected to:
#[doc = include_str!("../../CHANGELOG.md")]
pub mod changelog {}
With a symlink at core/core/CHANGELOG.md pointing to ../../CHANGELOG.md, the file is now included in the published package.
How the Bug Was Triggered (v0.59.0, Issue #8222)#
The include_str! line existed for some time but was dormant because the docs module was gated behind a docsrs cfg flag β it only compiled when building for docs.rs. In the older single-crate layout, lib.rs declares:
#[cfg(docsrs)]
pub mod docs;
PR #8183 ("docs: establish OpenDAL specifications") changed core/core/src/lib.rs to expose the module unconditionally β making pub mod docs; always compile. From that point on, cargo check against the published opendal-core = "0.59.0" tarball failed with :
error: couldn't read `.../opendal-core-0.59.0/src/docs/../../../CHANGELOG.md`:
The system cannot find the file specified. (os error 2)
--> .../opendal-core-0.59.0/src/docs/mod.rs:34:9
|
34 | #[doc = include_str!("../../../CHANGELOG.md")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This issue was resolved in PR #8223 by adding a CHANGELOG.md symlink at core/core/CHANGELOG.md pointing to ../../CHANGELOG.md and updating the include_str! path from ../../../CHANGELOG.md to ../../CHANGELOG.md.
Root Causes#
-
include_str!paths that escape the crate root. Any path that traverses above the crate's top-level directory (../../..from a deeply nested source file) will be valid in the monorepo but absent from the crates.io tarball. -
Unconditional module exposure. When a module containing cross-boundary
include_str!calls is only compiled under#[cfg(docsrs)], the build-time file resolution is deferred to docs.rs (where the full repo layout is available). Removing that guard surfaces the latent breakage for all users. -
Missing
includefield in Cargo.toml.core/core/Cargo.tomlusesexclude = ["/tests/"]but has noincludefield . Without an explicitinclude, only files inside the crate root are packed β cross-boundary file references are silently absent.
Resolution Patterns#
| Approach | How It Works | Used In OpenDAL? |
|---|---|---|
| Symlink the file into the crate root | Create core/CHANGELOG.md β ../CHANGELOG.md; shorten include_str! path to ../../CHANGELOG.md | Yes β applied in commit 63f74dcd for the v0.53 layout ; same pattern used for LICENSE/NOTICE in PR #7931; applied to fix the changelog issue in PR #8223 |
Gate module on #[cfg(docsrs)] | The include_str! is only evaluated on docs.rs where the full tree is present | Used in the v0.53 lib.rs; removed by PR #8183 for the new layout |
include field in Cargo.toml | Explicitly list out-of-crate files so cargo publish bundles them | Not yet applied to core/core/Cargo.toml as of v0.59.0 |
The symlink approach is the least invasive because cargo publish follows symlinks and includes their target content in the tarball. It keeps the include_str! path short and requires minimal source code changes.
Key Files#
core/core/src/docs/mod.rsβ contains the correctedinclude_str!("../../CHANGELOG.md")callcore/core/CHANGELOG.mdβ symlink pointing to../../CHANGELOG.md, added in PR #8223core/core/Cargo.tomlβ missingincludefield; only hasexclude = ["/tests/"]core/src/lib.rsβ#[cfg(docsrs)] pub mod docs;guard in the stable v0.53 layout- Issue #8222 β canonical bug report with exact error and reproduction steps
- PR #8183 β removed the
docsrsguard; triggered the regression - PR #8223 β resolved the issue by adding the changelog symlink and correcting the path
- PR #7931 β precedent: symlinks used to include LICENSE/NOTICE in split crates
Checklist for New Sub-Crate Splits#
When adding a new publishable sub-crate from the OpenDAL workspace:
- Audit every
include_str!in that crate for paths that escape the crate root (../..or deeper). - If the referenced file lives in a parent directory, either symlink it into the sub-crate root or copy it.
- If a module is doc-only (changelog, RFCs), keep it gated on
#[cfg(docsrs)]to avoid evaluatinginclude_str!in downstreamcargo check. - Consider adding an explicit
includefield toCargo.tomlto make packaging intent explicit and avoid silent omissions.