Credential and Secrets Management#

The SDK's credential system resolves, distributes, and forwards API keys and provider secrets into LLM calls automatically — no manual parameter threading required. Every incoming request passes through VaultMiddleware, which assembles secrets from two sources (environment variables and the remote vault API), stores them on request state, and makes them available to any code running within that request scope via a RoutingContext ContextVar.

Two secret kinds are supported 1:

• provider_key — standard provider API keys (OpenAI, Anthropic, Gemini, Cohere, etc.)
• custom_provider — custom deployments with structured credentials (AWS Bedrock, SageMaker, VertexAI, OpenAI-compatible endpoints)

Request-Scoped Secret Flow#

Incoming Request
    │
    ▼
VaultMiddleware.dispatch() sdk/agenta/sdk/middleware/vault.py
    │ 1. Reads {PROVIDER}_API_KEY env vars → local SecretDTOs
    │ 2. Calls GET /api/vault/v1/secrets → remote secrets
    │ 3. Deduplicates and caches by Authorization header
    │ → request.state.vault = {"secrets": [...]}
    │
    ▼
entrypoint.execute_wrapper() sdk/agenta/sdk/decorators/routing.py
    │ secrets = state.vault.get("secrets")
    │ → RoutingContext(secrets=secrets) [ContextVar, request-scoped]
    │
    ▼
SecretsManager.get_provider_settings(model) sdk/agenta/sdk/managers/secrets.py
       reads secrets from RoutingContext
       resolves api_key / extras for model
    → {"model": "...", "api_key": "...", ...} → litellm.completion(**kwargs)

1. VaultMiddleware runs on every request and writes to request.state.vault.
2. execute_wrapper extracts state.vault.get("secrets") 2 and injects it into a RoutingContext ContextVar 3, making secrets implicitly available throughout the request.
3. Any code calling SecretsManager.get_provider_settings(model) reads from that ContextVar via get_from_route() and gets back a dict of litellm-ready kwargs.

VaultMiddleware: Secret Fetching and Merging#

VaultMiddleware._get_secrets() performs two fetches on each request:

1. Local env-var secrets — iterates _PROVIDER_KINDS (derived from StandardProviderKind) and reads {PROVIDER}_API_KEY environment variables 4. Each found key is wrapped into a SecretDTO.

2. Remote vault secrets — calls GET /api/vault/v1/secrets with the request's Authorization header forwarded 5.

Merging and deduplication: Standard provider keys from the vault overwrite local env-var entries for the same provider; custom providers are collected in a separate list and appended 6. Local env-var secrets thus act as fallback defaults; vault entries take precedence for standard providers.

Caching: Results are stored in a TTLLRUCache keyed on the Authorization header hash. Caching can be disabled via the AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED environment variable 7.

SecretsManager: Credential Resolution#

SecretsManager in sdk/agenta/sdk/managers/secrets.py is the consumer-side API for credentials. The primary entry point is get_provider_settings(model):

1. Fetch — calls get_from_route() to read secrets from the current RoutingContext 8.
2. Parse — _parse_secrets() splits the raw list into provider_key and custom_provider entries.
3. Provider lookup — checks model_to_provider_mapping (a static catalog) for standard models; falls back to scanning custom_provider secrets for models that match custom deployment model lists 9.
4. Model normalization — _get_compatible_model() strips the provider_slug prefix and maps custom-kind deployments to the openai/ litellm prefix (since custom providers are OpenAI-compatible).
5. Credential extraction — for provider_key secrets, extracts api_key; for custom_provider secrets, merges the extras dict (containing api_base, api_version, AWS credential fields, etc.) directly into the return dict 10.

The returned dict is passed directly to litellm.completion() as **kwargs.

Custom Provider Support#

Custom providers (AWS Bedrock, SageMaker, VertexAI, or any OpenAI-compatible endpoint) are stored as custom_provider secrets. Their structure includes a provider_slug, a kind (e.g. bedrock, vertexai, custom), a url (base URL), and an extras dict for provider-specific auth fields.

_parse_custom_secrets() transforms the raw secret into a normalized dict, merging url into extras as api_base and version as api_version. The models field holds the list of model IDs supported by this deployment, which is how get_provider_settings matches a requested model to a custom secret.

Model ID convention: Custom models use the format {provider_slug}/{kind}/{model_name} (e.g. mybedrock/bedrock/us.anthropic.claude-haiku-4-5). _get_compatible_model() strips the slug prefix; if kind is custom, it additionally replaces {slug}/custom/ with openai/ since litellm treats OpenAI-compatible endpoints as the openai provider.

Bedrock specifics (post-PR #5057): Bedrock bearer tokens are routed to the AWS_BEARER_TOKEN_BEDROCK channel — not ANTHROPIC_API_KEY, which would cause mis-authentication on the direct Anthropic API 11. The frontend ConfigureProviderDrawer now includes a dedicated bearerToken field for Bedrock, with declarative either/or validation (bearer token OR access key + secret key) 11.

Known Issues and Recent Fixes#

PR #5057 — Credential Resolution Overhaul 11#

Three systemic failures were fixed in this PR:

Default agent templates now emit llm: {provider: "openai", model: ...} instead of a bare model, preventing bare-model failures in new agents 11.

Bug #4933 — Custom Provider Credentials Not Threaded 12#

A separate bug caused custom provider credentials to be silently ignored in the evaluator: get_provider_settings_from_workflow() was not being called in the relevant code path in handlers.py, causing fallback to gpt-4o-mini with no error 13. Fix: call the settings resolver in the evaluator loop.

Debugging Tips#

• Runner logs (post-PR #5057) print a consolidated line before each model execution showing resolved model, provider, deployment, connection, and secret key names (not values) — use this to verify which credentials were actually resolved 11.
• If credentials are silently missing, check: (1) AGENTA_SERVICE_MIDDLEWARE_CACHE_ENABLED isn't serving a stale cache, (2) the vault API returned 200, (3) get_provider_settings_from_workflow() is called in the relevant execution path.