Retrieval API (POST /api/v1/retrieval)#

POST /api/v1/retrieval retrieves ranked document chunks from one or more datasets without triggering LLM completion. This makes it the right entry point for conditional RAG workflows (retrieve → inspect → decide whether to generate), custom reranking pipelines, or any external tooling that needs raw retrieval results.

The handler is retrieval_test() in api/apps/restful_apis/chunk_api.py. Authentication uses a Bearer API key; the caller must own every dataset in dataset_ids. All datasets in a single request must share the same embedding model — mismatches are rejected with a DATA_ERROR 1.

Request Parameters#

POST /api/v1/retrieval
Content-Type: application/json
Authorization: Bearer <YOUR_API_KEY>

Required

Retrieval tuning

Filtering

Enrichment flags

Full parameter documentation is in the HTTP API reference under Retrieve chunks.

Response Format#

{
  "code": 0,
  "data": {
    "total": 4,
    "chunks": [
      {
        "id": "<chunk_id>",
        "content": "...",
        "document_id": "<doc_id>",
        "document_keyword": "<filename>",
        "similarity": 0.82,
        "term_similarity": 0.75,
        "vector_similarity": 0.91,
        "important_keywords": ["rag", "retrieval"],
        "highlight": "...<em>rag</em>...",
        "positions": [[1, 120, 40, 200, 60]],
        "image_id": "",
        "tag_kwd": []
      }
    ],
    "doc_aggs": [
      { "doc_id": "...", "doc_name": "paper.pdf", "count": 3 }
    ]
  }
}

Key fields 16:

• similarity — composite score blending term_similarity and vector_similarity at the configured weight.
• doc_aggs — per-document chunk counts, useful for building attribution UIs.
• positions — bounding-box coordinates [page, x0, y0, x1, y1] for PDF highlighting.
• Vectors are stripped before the response is sent 17; internal field names are remapped (e.g. content_with_weight → content, chunk_id → id) 18.

Internal Pipeline#

The handler delegates to settings.retriever.retrieval(), which maps to Dealer.retrieval() in rag/nlp/search.py — the full hybrid-search pipeline with BM25 + KNN scoring and reranking 19.

Call sequence inside retrieval_test():

1. Auth + model resolution — Validates dataset ownership; resolves embedding model and optional rerank model from tenant config 20.
2. Cross-language expansion (if cross_languages) — Translates question via cross_languages() 12.
3. Keyword extraction (if keyword=true) — Calls the tenant's chat LLM via keyword_extraction() and appends the result to question 10.
4. Metadata pre-filter (if metadata_condition and no explicit document_ids) — Resolves matching doc_ids via DocMetadataService before the vector search 9.
5. Core retrieval — Dealer.retrieval() executes hybrid BM25+KNN search, post-retrieval reranking, and pagination 19.
6. ToC enhancement (if toc_enhance) — Re-ranks with table-of-contents chunks via retrieval_by_toc() 14.
7. Child chunk expansion — retrieval_by_children() replaces parent summary chunks with their children 21.
8. Knowledge-graph injection (if use_kg) — Prepends a KG-synthesized chunk at index 0 13.

See Hybrid Search and Retrieval for scoring details (BM25/KNN weights, reranking paths, similarity threshold behavior).

Consumers and Integrations#

MCP server — The MCP retrieval() tool posts directly to /api/v1/retrieval. When dataset_ids is omitted, it auto-resolves all accessible datasets via resolve_dataset_ids(). Results are enriched with a two-level TTL cache (32-entry LRU, ~300 s TTL) for dataset and document metadata; use force_refresh: true to bypass it 22.

Dify integration — A separate Dify-compatible endpoint at POST /api/v1/dify/retrieval mirrors the same retrieval pipeline but uses Dify's request schema (knowledge_id, retrieval_setting.top_k, etc.) rather than RAGFlow's native schema.

SDK / direct HTTP — Any HTTP client can call this endpoint independently of the chat API. This is the standard pattern for building retrieval-only tools, evaluation harnesses, or agent components that need to conditionally decide whether to generate based on retrieved context.