Parent-Child Retrieval Architecture#

Parent-child retrieval is a hierarchical RAG strategy where child chunks are embedded and indexed in the vector store, while parent chunks are stored in the database unembedded. At query time, semantic search operates over the fine-grained child chunks, but the system returns the broader parent context to the LLM. This gives the precision of small-chunk similarity search without sacrificing the coherence of larger context windows.

The index structure is identified by IndexStructureType.PARENT_CHILD_INDEX and is implemented in ParentChildIndexProcessor.

Indexing Pipeline#

ParentChildIndexProcessor.transform() supports two parent modes (controlled by the ParentMode enum):

• PARAGRAPH — the document is first split into paragraph-sized parent chunks; each parent is then further split into child chunks via _split_child_nodes(). Parents get their own UUID doc_id; children get separate UUIDs 1.
• FULL_DOC — the entire document is concatenated into a single parent; child chunks are subsets of that parent 2.

Only child chunks go into the vector store. load() calls vector.create(formatted_child_documents) on the child list — parents are never embedded 3.

Persistence to the database is handled by DatasetDocumentStore.add_documents(save_child=True):

• Parent documents are written to the document_segments table as DocumentSegment rows 4.
• Each child is written to the child_chunks table as a ChildChunk row with a segment_id foreign key pointing to its parent, and a sequential position field 5.

Query-Time: Child-to-Parent Reconstruction#

All retrieval methods (semantic, full-text, hybrid) return child chunk doc_ids from the vector store. The reconstruction happens in RetrievalService.format_retrieval_documents().

Vector search → child doc_ids → ChildChunk rows (by index_node_id) → segment_id → DocumentSegment (parent)

The key steps:

1. Route by doc_form: documents with doc_form == PARENT_CHILD_INDEX populate child_index_node_ids instead of index_node_ids 6.
2. Child → parent lookup: ChildChunk rows are fetched by index_node_id; their segment_id values are collected to identify which parent segments to load 7.
3. Parent segment fetch: parent DocumentSegment rows are queried by segment_id, filtered to enabled=True and status='completed' 8.
4. Max-score aggregation: for each parent segment, the system iterates over all matching child chunks and assigns max_score = max(child.score for all matched children) 9. This score is stored in a SegmentChildMapDetail dict 10.
5. Final sort: results are sorted by max_score descending before returning 11. Child chunks are also sorted by score descending within each parent 12.

Data Flow Diagram#

Key Types & Data Models#

Cleanup#

ParentChildIndexProcessor.clean() handles deletion:

• Resolves child index_node_ids from the child_chunks table via a segment_id join 13.
• Deletes child vectors from the vector store via vector.delete_by_ids(child_node_ids) 14.
• Optionally deletes ChildChunk DB rows when delete_child_chunks=True 15.
• Supports a precomputed_child_node_ids kwarg to avoid race conditions when segments are deleted before the cleanup runs 16.