Weaviate Vector Store#

The Weaviate backend (WeaviateVector) is implemented in weaviate_vector.py as part of Dify's pluggable vector store layer. It manages a single Weaviate collection per dataset, using a module-level singleton client with thread-safe initialization 1 and Redis locking to prevent concurrent collection creation races 2.

Key schema properties stored per object 3:

Tokenization is configurable via WEAVIATE_TOKENIZATION; defaults to WORD 4.

UUID Generation and the Collision Problem#

_get_uuids() generates deterministic UUID5 values using the URL namespace (6ba7b811-…) hashed against page_content only. This means two segments from different documents with identical text produce the same Weaviate object UUID, causing silent overwrites during batch insert.

This was reported in issue #36137 and addressed by PR #36140, which proposed prepending doc_id to the hash input:

# Before
uuid_val = uuid5(URL_NAMESPACE, doc.page_content)

# After (PR #36140)
doc_id = (doc.metadata or {}).get("doc_id", "")
text_to_hash = f"{doc_id}_{doc.page_content}" if doc_id else doc.page_content
uuid_val = uuid5(URL_NAMESPACE, text_to_hash)

Note: As of issue #39175, the current code at _get_uuids() 5 still hashes doc.page_content only — the PR #36140 fix may not yet be merged into main. Verify the live state of this method before relying on collision-safe behavior.

Deletion Patterns: UUID-Based vs. Metadata-Based#

This is the most subtle area of the Weaviate implementation. Dify's BaseVector contract uses logical segment IDs (index_node_id, a UUID4) as the ids passed to delete_by_ids(). Weaviate, however, stores objects under content-derived UUID5s — not the logical IDs. The mismatch means delete_by_ids(ids) silently no-ops because the UUID4 segment IDs don't match any Weaviate object UUIDs.

delete_by_ids (UUID-based)#

delete_by_ids() calls col.data.delete_by_id(uid) per ID and ignores 404s. When the IDs passed are logical index_node_id values (UUID4) rather than content-derived UUID5s, no object is found and nothing is deleted — but no error is raised either. This is the root cause of vectors surviving segment disable/delete operations 6.

delete_by_metadata_field (metadata-based)#

delete_by_metadata_field(key, value) uses Weaviate's filter API to bulk-delete all objects matching a property value:

col.data.delete_many(where=Filter.by_property(key).equal(value))

Calling this with key="doc_id" and value=index_node_id correctly targets the stored metadata field, bypassing the UUID mismatch entirely.

The Fix Applied by PRs #37068 and #37384#

Both PR #37068 and PR #37384 switched child chunk vector refresh logic in vector_service.py from:

vector.delete_by_ids(delete_node_ids) # broken for Weaviate

to:

for node_id in delete_node_ids:
    vector.delete_by_metadata_field("doc_id", node_id) # correct

PR #37384 also fixed a secondary issue: index_node_hash was not being recomputed when chunk content changed, causing the duplicate-check to see the old hash and block re-insertion even after a stale vector was removed 7.

Open Bug: Segment Disable/Delete Leaves Orphaned Vectors#

Issue #39175 (filed 2026-07-17, currently open) documents that disabling or deleting a document segment does not remove its Weaviate vector. The delete_by_ids() path is still used for parent segment cleanup (as opposed to child chunks), so the UUID mismatch persists for top-level segment vectors. The correct fix follows the same pattern: replace delete_by_ids with delete_by_metadata_field("doc_id", segment_id) wherever Weaviate objects need to be removed by logical segment identity.

Quick Reference#

Entry points for further reading:

• WeaviateVector class
• _get_uuids() — collision-prone UUID generation
• delete_by_ids() — UUID-based deletion (problematic for logical IDs)
• delete_by_metadata_field() — metadata-based deletion (correct pattern)
• Issue #39175 — active bug report for segment-level deletion failure
• PR #36140 — UUID collision fix (doc_id in hash)
• PR #37068 / PR #37384 — metadata-deletion fix for child chunks