Model Provider Architecture#

Overview#

Introduced in v0.27.0 1, the model provider architecture replaces the legacy flat tenant_llm table with a normalized three-tier hierarchy: Provider → Instance → Model. This enables multiple configured instances of the same provider (e.g., two different OpenAI API keys), multi-capability models expressed as integer bit flags, and group-based routing/load balancing across instances.

Table Hierarchy#

TenantModelProvider (one per provider per tenant)
  └── TenantModelInstance (one per API key / endpoint config)
        └── TenantModel (one per model name, with bit-flag capabilities)

TenantModelGroup / TenantModelGroupMapping (optional routing layer over TenantModel rows)

TenantModelProvider#

tenant_model_provider — scoped by a (tenant_id, provider_name) unique index. Maps a tenant to a named provider (e.g., "OpenAI").

TenantModelInstance#

tenant_model_instance — holds credentials and config for one concrete API configuration: api_key, instance_name, status, and a JSON extra blob (for custom base URLs etc.).

TenantModel#

tenant_model — the leaf node. Each row represents one model (e.g., gpt-4o) within an instance. Key fields:

• model_name — model identifier string
• provider_id / instance_id — VARCHAR(32) UUID foreign keys (not integers)
• model_type — integer bit flags encoding one or more capabilities (see below)
• extra — JSON blob for per-model overrides (max tokens, etc.)

TenantModelGroup / TenantModelGroupMapping#

tenant_model_group + tenant_model_group_mapping — optional routing layer. A group references multiple (provider_id, instance_id, model_id) triples with per-member weight values and a strategy (default: "weighted").

Model Type Bit Flags#

TenantModel.model_type is an IntegerField where each bit represents a capability, defined in ModelTypeBinary. A single row can express multiple capabilities with bitwise OR:

Utilities in api/utils/model_utils.py:

• calculate_model_type(names) — converts a list of type name strings → integer flag. Handles legacy aliases: speech2text → asr, image2text → vision.
• get_model_type_human(flag) — decodes an integer flag back to a list of lowercase type names.

FK References in Downstream Tables#

After v0.27.0, tenant, knowledgebase, dialog, and memory carry both the legacy string *_id columns and new tenant_*_id VARCHAR(32) columns that hold tenant_model.id UUIDs 2 3 4 5:

Service Layer#

CRUD — api/db/services/tenant_model_service.py#

Low-level Peewee queries scoped to TenantModel: filter by provider_id, instance_id, and model_type (using bitwise & checks); batch status/type updates; delete by instance.

Business Logic — api/db/joint_services/tenant_model_service.py#

High-level resolution used by API services and task executors:

• get_tenant_default_model_by_type() — returns the tenant's default tenant_model row for a capability type
• resolve_model_config() — resolves a model reference (UUID or composite name@instance@provider string) into a provider config dict
• get_model_config_by_id() — fetches config by tenant_model.id with tenant ownership validation
• ensure_tenant_model_ids_for_params() — auto-populates tenant_*_id FK fields in request parameters from model name lookups
• split_model_name() — right-anchored parser for composite model name strings

Legacy tenant_llm Table#

The original tenant_llm table (composite PK on tenant_id, llm_factory, llm_name; model_type as a plain string) is retained for backward compatibility. New code should use the three-tier tables. Legacy rows in tenant_llm were migrated to the new tables by the mysql_migration.py stages 6.

The migrate_model_type_names() helper in db_models.py keeps the model_type strings in tenant_llm in sync by renaming speech2text → asr and image2text → vision at startup 7.

Migration Path#

Seven ordered stages in tools/scripts/mysql_migration.py, orchestrated by tools/scripts/run_migrations.sh, handle the full upgrade from tenant_llm to the new tables 6:

See the Database Migrations article for full details on running and extending the migration system.