Lambda MicroVM Sandbox#

Overview#

lambdaMicrovm.ts is the production sandbox provider for the in-app agent feature. It wraps AWS Lambda MicroVMs via @aws-sdk/client-lambda-microvms to give each agent conversation an isolated, stateful compute environment. The file lives at packages/shared/src/in-app-agent/server/sandbox/providers/lambdaMicrovm.ts (commit b97e819; later moved to web/src/features/in-app-agent/server/sandbox/providers/lambdaMicrovm.ts in PR #15492).

The provider implements the SandboxProvider interface, which defines three top-level operations: ensureSession, suspendSession, and terminateSession.

Session Lifecycle#

All session state is held in an in-memory Map<string, LambdaMicrovmSession> scoped to the provider instance. Each entry tracks the MicroVM's endpoint, an optional cached authToken, and the current toolCallFiles snapshot.

ensureSession#

ensureSession is the core entry point. When called with a conversationId and an optional existing sessionId, it:

1. Looks up the existing MicroVM via GetMicrovmCommand. If found and SUSPENDED, issues a ResumeMicrovmCommand first.
2. Waits for the HTTP bridge inside the MicroVM to become ready (waitForBridge, 30 s timeout, 500 ms polling interval).
3. Creates a new MicroVM via RunMicrovmCommand if no existing session is found.

Key timing constants 1:

• DEFAULT_SUSPEND_AFTER_IDLE_SECONDS = 60 — auto-suspend after 1 minute of idle
• DEFAULT_TERMINATE_AFTER_SUSPEND_SECONDS = 8 * 60 * 60 — terminate 8 hours after suspension
• DEFAULT_MAXIMUM_DURATION_SECONDS = 3_600 — hard 1-hour cap per session

New MicroVMs are launched with autoResumeEnabled: true and clientToken: randomUUID() for idempotency.

suspendSession#

Calls SuspendMicrovmCommand then removes the session from the in-memory map. ResourceNotFoundException / HTTP 404 errors are silently swallowed (session already gone).

terminateSession#

Calls TerminateMicrovmCommand. The in-memory entry is removed before the AWS call so the session is treated as gone even if the API call fails. Same 404-tolerance as suspend.

Operations & Authentication#

executeOperation dispatches the four supported operation types — read, write, edit, bash — to the MicroVM's HTTP bridge at {endpoint}/sandbox (port 5000) 1.

Each request carries:

• X-aws-proxy-port: 5000
• X-aws-proxy-auth: <token> — a short-lived token from CreateMicrovmAuthTokenCommand

Tokens are cached per session and refreshed proactively when within AUTH_TOKEN_REFRESH_BUFFER_MS = 60_000 ms of expiry (token lifetime is DEFAULT_AUTH_TOKEN_EXPIRATION_MINUTES = 30) 1.

Every operation also sends the current toolCallFiles snapshot in the POST body. This is the vector for a known race condition (see Known Issues below).

Optional Egress Network Connector#

If egressNetworkConnectorArn is provided (set via LANGFUSE_IN_APP_AGENT_SANDBOX_AWS_LAMBDA_MICROVM_EGRESS_NETWORK_CONNECTOR_ARN), it is passed as egressNetworkConnectors in RunMicrovmCommand to enforce network egress policy 2.

Health Check / Bridge Readiness#

waitForBridge polls {endpoint}/health with a fresh auth token on every attempt 1:

• Outer timeout: BRIDGE_READY_TIMEOUT_MS = 30_000 ms
• Poll interval: 500 ms
• Aborts immediately if the MicroVM transitions to TERMINATING or TERMINATED

Note on timeout handling: The current waitForBridge and executeOperation fetch() calls use no AbortSignal. This means a hung health probe or operation request will never be cancelled by the caller. If adding timeouts, prefer AbortSignal.timeout(ms) (Node 18+) for one-liners, or AbortController + clearTimeout for cases where you also need to abort on external signals. A related production incident (PR #14876) demonstrated that GC-fragile AbortController patterns with AbortSignal.timeout() on derived signals can silently lose their timeout — always pass the originating signal, not a derived one.

Known Issues#

Global tool_calls Directory Race (Issue #15954)#

The sandbox runtime server uses a single global /workspace/tool_calls directory. Because executeOperation sends toolCallFiles with every request and the runtime deletes/recreates that directory atomically, concurrent sandbox calls can destroy each other's files 3.

Affected code paths:

• packages/in-app-agent-sandbox-runtime/src/server.ts:71-85 — syncToolCallFiles nukes and recreates the global directory
• packages/in-app-agent-sandbox-runtime/src/server.ts:135-145 — every /sandbox request triggers the sync
• lambdaMicrovm.ts lines sending toolCallFiles on every executeOperation call 3

Fix direction: Use request/session-scoped directories instead of a single global one, or serialize per-session operations.

Configuration & Environment#

Key Source References#