Media Context Configuration#

Media context configuration controls how much surrounding text (measured in tokens) is attached to image and table chunks during document parsing. This context helps retrieval by enriching otherwise sparse media chunks with nearby prose. The setting is stored in parser_config under three related fields and is applicable to the Naive (General) and Manual parsers.

parser_config Fields#

On save, the frontend fans all three out from the same slider value, keeping them in sync 1. On load, the dialog reads image_table_context_window first and falls back to image_context_size then table_context_size for backwards-compatibility 2.

Frontend UI#

The setting surfaces in two places:

1. Dataset settings page — The ImageContextWindow component renders a slider (0–256) bound to parser_config.image_table_context_window. It is shown only when the Naive chunk method is selected 3.

2. Per-document chunk-method dialog — ChunkMethodDialog includes the same ImageContextWindow component for Naive documents 3. Its Zod schema validates image_table_context_window as a coerced number 4.

On submit (ChunkMethodDialog.onSubmit), all three fields are written simultaneously 5:

• image_table_context_window = the slider value
• image_context_size = the slider value
• table_context_size = the slider value

The dataset-level SavingButton does the same 1.

Backend Chunking — How the Settings Are Applied#

The two backend fields are consumed differently depending on the document type and parser. There are four distinct code paths:

DOCX — naive_merge_docx (Naive parser)#

naive.py reads both fields and passes them into naive_merge_docx 6 7:

table_context_size = max(0, int(parser_config.get("table_context_size", 0) or 0))
image_context_size = max(0, int(parser_config.get("image_context_size", 0) or 0))
chunks, images = naive_merge_docx(sections, …, table_context_size, image_context_size)

Inside naive_merge_docx, the _add_context() helper walks neighboring chunks and populates context_above / context_below fields using separate token budgets for tables vs. images 8. Both are honored independently.

PDF — append_context2table_image4pdf (Naive parser)#

For PDF files, naive.py calls 9:

if table_context_size or image_context_size:
    tables = append_context2table_image4pdf(sections, tables, image_context_size)

⚠️ Known inconsistency: append_context2table_image4pdf accepts only a single table_context_size parameter but is called here with image_context_size 10. The function applies that value as context for tables only (no separate image path). Image chunks in PDFs do not receive context through this path. The guard condition (if table_context_size or image_context_size) means any non-zero value triggers the function, but the effective budget is always image_context_size regardless of table_context_size.

PDF + DOCX — attach_media_context (Manual parser)#

rag/app/manual.py uses a third approach, calling attach_media_context post-tokenization for both PDF and DOCX :

table_ctx = max(0, int(parser_config.get("table_context_size", 0) or 0))
image_ctx = max(0, int(parser_config.get("image_context_size", 0) or 0))
if table_ctx or image_ctx:
    attach_media_context(res, table_ctx, image_ctx)

attach_media_context honors both image_context_size and table_context_size as separate budgets, selecting the nearest text chunk by page-position geometry or sequential proximity 12.

DSL Pipeline — TokenChunker#

The TokenChunker process node (used in agent/flow pipelines) stores table_context_size and image_context_size as first-class parameters in TokenChunkerParam 13 and applies them via _attach_context_to_media_chunks , which walks neighbors directionally using token budgets per chunk type.

Summary of Inconsistencies#

The PDF path in naive.py is the outlier: it calls append_context2table_image4pdf(sections, tables, image_context_size), passing image_context_size into the function's table_context_size parameter. The table_context_size value from config is effectively ignored for PDFs in the Naive parser. Since the UI sets both fields to the same value, this is usually benign in practice but is architecturally inconsistent.

Key Source Files#