Raw bun test mis-loads Playwright e2e specs and reports phantom failures#
Problem#
Running bun test (the bare command) in this repo collects and tries to execute the Playwright specs under e2e/*.spec.ts with Bun's own test runner. Playwright's test() refuses to run outside the Playwright runner, so those files error out — producing a scary "15 fail / 15 errors" summary that looks like a real regression but is purely a test-runner mismatch.
Symptoms#
$ bun test
error: Playwright Test did not expect test() to be called here.
...
at .../e2e/theme.spec.ts:21:1
276 pass
15 fail
15 errors
Ran 291 tests across 46 files.
The 15 failures/errors all originate from files under e2e/; the 276 passes are the real src/ unit + integration suite.
What Didn't Work#
- Reading the failures as a code regression. After a value-preserving CSS-token rename, seeing 15 failures immediately after the change suggests the change broke something. It did not — none of the failing files are exercised by the change, and the failure text is a runner error, not an assertion failure.
- Assuming
bun testis the project's test command. It is the Bun default, but this repo does not use it directly; the real suites are wrapped in scoped scripts.
Solution#
Use the project's scoped scripts, never bare bun test:
// package.json
"scripts": {
"test": "bun test src", // unit + integration only (Postgres-gated on DATABASE_URL)
"test:e2e": "playwright test" // Playwright runner; Docker/Testcontainers required
}
- Unit / integration:
bun run test(scoped tosrc/, soe2e/is never collected). - End-to-end:
bun run test:e2e(the Playwright runner). - Full gate:
just ci-check(lint, format-check, typecheck, pre-commit-run,test,test-e2e) — the mandatory pre-commit gate.
Run through the scoped scripts and the phantom failures disappear (276 pass / 0 fail); the e2e suite runs green under Playwright (24 passed).
Why This Works#
bun test <path> globs every *.test.ts / *.spec.ts it can find under the given path. Bare bun test defaults to the whole repo, which sweeps in e2e/*.spec.ts — Playwright specs that must run under the playwright binary, not Bun. Scoping to src (bun test src) excludes e2e/ by path, and playwright test owns the e2e specs with the correct runner and its Testcontainers/Docker setup. The two runners never see each other's files.
Prevention#
- Always invoke the scoped scripts (
bun run test,bun run test:e2e) or the fulljust ci-check— treat barebun testas "wrong tool" in this repo. - When a post-change test run shows failures, check the failure type before the count. A
Playwright Test did not expect test() to be called here(or any "runner X did not expect Y") message is a collection/runner mismatch, not a code regression — look at which runner loaded which files before diagnosing the code. - This mirrors the repo's testing rule: integration/e2e run via Testcontainers under Playwright; see AGENTS.md and [[e2e-dotenv-mise-clobbers-launcher-env]] for the e2e harness details.