Vector Database Plugin Architecture#

Dify's vector database (VDB) system is fully pluggable: each backend is a standalone Python workspace package under api/providers/vdb/ that advertises itself via a dify.vector_backends importlib entry point. The core RAG layer discovers and loads backends at runtime — no import-time registration, no monolithic switch statement.

30 backends ship out-of-the-box 1: Milvus, Qdrant, pgvector, Weaviate, Chroma, Elasticsearch, and more. The VectorType StrEnum lists all recognized type strings 2.

Key Components#

How Discovery Works#

get_vector_factory_class(vector_type) is the single entry point for loading a backend:

1. Cache check — returns the in-process cached class if already loaded 3.
2. Entry-point scan — iterates entry_points(group="dify.vector_backends") and matches by ep.name == vector_type 4.
3. Builtin fallback — checks _BUILTIN_VECTOR_FACTORY_TARGETS (currently empty dict; reserved for test stubs) 5.
4. Error — if nothing matched, raises ValueError listing all installed backends 6.

After uv sync, each installed package's dist-info contains the entry-point metadata. The registry never imports a backend package unless that backend type is actually requested.

Plugin Package Structure#

Each providers/vdb/vdb-<name>/ directory is a uv workspace member 7. Its pyproject.toml must declare:

• Dependencies — backend-specific client library (e.g. weaviate-client==4.22.0 8, pymilvus~=2.6.12 9)
• Entry point — one line mapping the VectorType string to the factory class:

[project.entry-points."dify.vector_backends"]
weaviate = "dify_vdb_weaviate.weaviate_vector:WeaviateVectorFactory"

10

The factory class must implement AbstractVectorFactory.init_vector(dataset, attributes, embeddings) → BaseVector.

BaseVector Contract#

All backends implement BaseVector (abstract methods):

Dependency Management#

api/pyproject.toml uses a uv workspace 7. The default-groups list includes vdb-all 11, so a bare uv sync installs all 30 backends. To install a single backend:

uv sync --group vdb-milvus # installs only dify-vdb-milvus

Individual groups are defined at 12. After adding or modifying a plugin package, run uv sync to regenerate dist-info so entry points match pyproject.toml 13.

Runtime Selection#

Vector._init_vector() determines which backend to use:

1. If the dataset already has an index_struct_dict, its type field is used (e.g., the collection was created with a specific backend).
2. Otherwise, falls back to dify_config.VECTOR_STORE (the VECTOR_STORE environment variable).

The Vector class also uses a _LazyEmbeddings proxy 14 so delete/cleanup paths never trigger billing-API calls just to instantiate the embedding model.

Adding a New Backend#

1. Create api/providers/vdb/vdb-<name>/ with a pyproject.toml declaring the entry point under dify.vector_backends.
2. Implement AbstractVectorFactory and BaseVector subclasses.
3. Add dify-vdb-<name> to the workspace sources and dependency groups in api/pyproject.toml.
4. Add the type string to VectorType.
5. Run uv sync.

Key reference files:

• api/core/rag/datasource/vdb/vector_backend_registry.py — discovery logic
• api/core/rag/datasource/vdb/vector_factory.py — Vector facade and AbstractVectorFactory
• api/core/rag/datasource/vdb/vector_base.py — BaseVector contract
• api/core/rag/datasource/vdb/vector_type.py — VectorType enum
• api/providers/vdb/vdb-weaviate/pyproject.toml — example plugin manifest
• api/pyproject.toml — workspace and dependency group declarations