Document Engine Resource Cleanup#
When a dataset (knowledge base) or tenant (user) is deleted in RAGFlow's Go service layer, chunk-store resources must be explicitly cleaned up in the active document engine. Because engine operations cannot participate in the SQL transaction, cleanup always runs before the database transaction that removes the rows. Failures during engine cleanup block the deletion so the operation can be retried without leaving orphaned data.
The three supported engines โ Elasticsearch, Infinity, and OceanBase โ have fundamentally different storage shapes, so deletion logic is engine-specific and is dispatched through the DocEngine interface.
Engine Storage Models and Naming#
| Engine | Chunk store shape | Name pattern |
|---|---|---|
| Elasticsearch | One shared index per tenant; all KBs live inside it | ragflow_{tenant_id} |
| Infinity | One table per dataset within the tenant namespace | ragflow_{tenant_id}_{kb_id} |
| OceanBase | Shared physical table per tenant; rows are scoped by kb_id | ragflow_{tenant_id} |
The ragflow_{tenant_id} prefix is produced by IndexName(uid). A parallel memory_{tenant_id} index is used for memory/conversation chunks and follows the same engine-specific deletion pattern.
Each engine also maintains a metadata store for per-document metadata (ragflow_doc_meta_{tenant_id} on ES/OceanBase; a dedicated table on Infinity), accessed via DropMetadataStore / DeleteMetadata.
Dataset Deletion#
deleteDataset in internal/service/dataset/crud.go orchestrates the full teardown for one knowledge base. Engine cleanup is delegated to deleteDatasetEngineData:
- Collect document IDs from the database before the transaction begins.
- Call
docEngine.DeleteChunks(ctx, {"doc_id": docIDs}, "ragflow_{tenantID}", kb.ID)to delete all chunks belonging to those documents . - Call
docEngine.DeleteMetadata(ctx, {"doc_id": docIDs}, kb.TenantID)to remove per-document metadata entries .
DeleteChunks targets the specific rows via a filter โ it does not drop the store itself. The shared index/table is preserved so other KBs in the tenant are unaffected.
Note:
DropChunkStoreis also exposed (e.g. fromTenantService.DeleteChunkStore) for explicit table/index tear-down, but the dataset deletion path usesDeleteChunksto avoid destroying the shared structure.
Tenant / User Deletion#
Tenant deletion requires a different strategy because the entire chunk store and metadata index must be removed. The logic lives in deleteExternalData (internal/admin/user_delete.go) and dispatches through dropUserChunkStore.
For datasets owned by the deleted tenant:
dropUserChunkStore(ctx, docEngine, IndexName(ownedTenantID), ownedDatasetIDs)
dropUserChunkStore(ctx, docEngine, MemoryIndexName(ownedTenantID), memoryIDs)
docEngine.DropMetadataStore(ctx, ownedTenantID)
dropUserChunkStore is engine-aware:
- Infinity: iterates over every dataset ID and calls
DropChunkStore(ctx, indexName, id)for each, because each dataset is its own table. - Elasticsearch / OceanBase: calls
DropChunkStore(ctx, indexName, "")once, because those engines use a shared structure under a single name.
For datasets the deleted user created on a foreign tenant (a tenant they joined but don't own), the code cannot drop the shared index. Instead it calls DeleteChunks(ctx, {"kb_id": dataset.ID}, indexName, dataset.ID) and, for individual documents outside deleted datasets, DeleteChunks filtered by doc_id .
Engine-Specific DropChunkStore Behavior#
Elasticsearch#
DropChunkStore delegates to dropIndex(ctx, baseName), which issues an ES index-delete for the full tenant index. The datasetID argument is ignored โ the entire ragflow_{tenant_id} index is dropped .
Infinity#
DropChunkStore + dropTable drops the table named {baseName}_{datasetID}, i.e. ragflow_{tenant_id}_{kb_id}. Existence is checked first; a missing table returns an error from dropTable but callers in the tenant-deletion path iterate per-dataset so each drop is isolated. The table name is constructed by buildChunkTableName.
OceanBase#
DropChunkStore is scope-aware:
- With a non-empty, non-
"skill"datasetID: issuesDELETE FROM {table} WHERE kb_id = ?(ormemory_id = ?for memory tables), leaving the shared table intact. - With an empty
datasetIDor"skill": issuesDROP TABLE IF EXISTS {table}, removing the whole table.
This means an unscoped call (tenant-level teardown) destroys the OceanBase table, while a dataset-level call prunes only that dataset's rows.
Key Files#
| File | Role |
|---|---|
internal/engine/engine.go | DocEngine interface โ all engine methods |
internal/service/dataset/crud.go | Dataset deletion + deleteDatasetEngineData |
internal/admin/user_delete.go | Tenant/user deletion, deleteExternalData, dropUserChunkStore |
internal/engine/elasticsearch/chunk.go | ES DropChunkStore |
internal/engine/infinity/chunk.go | Infinity CreateChunkStore / DeleteChunks |
internal/engine/infinity/common.go | Infinity dropTable, buildChunkTableName |
internal/engine/oceanbase/schema.go | OceanBase DropChunkStore (scoped vs. full drop) |
internal/service/tenant.go | DeleteChunkStore / DeleteMetadataStore service wrappers |