Knowledge Base API#

Dify exposes two parallel sets of REST endpoints for managing knowledge bases (datasets): the Console API (browser/dashboard, under /console/api/datasets/) and the Service API (external integrations, under /v1/datasets/). Both surface the same core capabilities — document upload, metadata field management, and document metadata value assignment — but differ in auth (session-based vs. API-key) and minor response shapes.

Key source files:

• Service API document endpoints: api/controllers/service_api/dataset/document.py
• Service API metadata endpoints: api/controllers/service_api/dataset/metadata.py
• Console metadata endpoints: api/controllers/console/datasets/metadata.py
• Business logic: api/services/metadata_service.py
• Request/response entities: api/services/entities/knowledge_entities/knowledge_entities.py

Document Upload and Configuration#

All document CRUD operations are available on the Service API:

Deprecation note: Legacy underscore routes (create_by_text, create_by_file) remain registered for backward compatibility but are marked deprecated 7.

Key configuration fields (DocumentTextCreatePayload)#

8

For file uploads the same fields are passed as a JSON string in the data form field 13.

Indexing status tracking#

Document creation is asynchronous. The response includes a batch ID; poll GET /datasets/{dataset_id}/documents/{batch}/indexing-status to track progress through stages: waiting → parsing → cleaning → splitting → indexing → completed 14.

Getting a document#

GET /datasets/{dataset_id}/documents/{document_id} accepts a metadata query param 15:

• all (default) — full document fields + metadata
• only — returns only id, doc_type, doc_metadata
• without — omits doc_type and doc_metadata

Metadata Field Management#

Metadata fields are schema-level definitions attached to a knowledge base. They allow documents to be annotated with typed structured values that can later be used for filtering during retrieval (see Knowledge Base Metadata Filtering).

Endpoints#

Both the Console and Service APIs expose the same operations 16 17:

The Console metadata endpoints are gated by @enterprise_license_required 18.

Enterprise license requirement#

In self-hosted enterprise deployments (DEPLOYMENT_EDITION=ENTERPRISE), all Service API endpoints — including the Knowledge Base API under /v1/datasets/ — require a valid enterprise license. When the enterprise license is invalid (INACTIVE, EXPIRED, or LOST status) or unavailable, requests to /v1 endpoints return 403 Forbidden with the error marker license_required. This global check is enforced by the request gate in app_factory.py before controller-level auth or validation runs.

Note: ENTERPRISE_ENABLED has been deprecated and replaced with DEPLOYMENT_EDITION as the single source of truth for edition checking (set in api/configs/deploy/__init__.py).

Custom field schema (MetadataArgs)#

20

{
  "type": "string" | "number" | "time",
  "name": "<field_name_max_255_chars>"
}

MetadataService.create_metadata validates that the name is ≤ 255 characters, is not a duplicate, and does not conflict with built-in field names 21.

Built-in fields#

Five system-defined fields are always available via MetadataService.get_built_in_fields() 22:

• document_name
• uploader
• upload_date
• last_update_date
• source

Enabling built-in fields for a KB auto-populates them on all existing documents. Disabling removes the values from all documents 23.

Setting Document Metadata Values#

To assign values to metadata fields on specific documents, use the batch update endpoint:

Service API returns {"result": "success"} with HTTP 200 24; Console API returns 204 No Content 25.

Request body (MetadataOperationData)#

26

{
  "operation_data": [
    {
      "document_id": "<doc_uuid>",
      "metadata_list": [
        { "id": "<metadata_field_id>", "name": "<field_name>", "value": "<value>" }
      ],
      "partial_update": false
    }
  ]
}

• operation_data — array of per-document operations (DocumentMetadataOperation) 27
• metadata_list — each entry is a MetadataDetail with id, name, and value (string, number, or null) 28
• partial_update (default false) — when true, only the listed fields are updated; when false, the document's metadata is replaced entirely

MetadataService.update_documents_metadata acquires per-document Redis locks to prevent concurrent modifications 29.

Metadata in retrieval#

Document metadata values are stored in the doc_metadata JSON column on DatasetDocument. During retrieval, metadata filtering conditions are translated into SQLAlchemy predicates against this column — see Knowledge Base Metadata Filtering for supported operators and filter structure.