Chemical Notation OCR#

Chemical notation — including molecular formulas, structural diagrams, and reaction equations — is a known area of reduced OCR accuracy in Docling, with the sharpest failures occurring when chemical structure images appear alongside text in table cells. The problems split into two categories:

1. General misrecognition of chemical symbols by the underlying OCR engine (EasyOCR, RapidOCR)
2. Table-context failures where a chemical structure image in one cell causes content to bleed into adjacent cells

The canonical issue is docling #1299: Multi-line cell table with image fails to OCR correctly.

Table Cell Bleeding: The Primary Failure Mode#

The most impactful failure is cell content bleeding: a chemical structure image in one table column contaminates the OCR output of a neighboring text column. Documented examples from pharmaceutical tables 1:

Root cause: TableFormer's cell-grid binding requires a ≥30% bounding-box overlap (textcell_overlap = 0.3) between a predicted table cell and a page text region 2. When a cell contains an embedded chemical structure image, the image bbox and the text bbox overlap ambiguously, causing misassignment of content across cells.

Switching between RapidOCR and EasyOCR shifts which cells are wrong but does not eliminate the errors 1. Toggling do_code_enrichment and do_formula_enrichment also does not resolve the issue 1.

How Docling Handles Chemical Content#

Docling does have explicit chemical notation support, but it applies to standalone chemical elements detected by the layout model — not to chemical structures embedded inside table cells.

• The Chandra OCR parser recognizes "Chemical-Block" as a distinct layout label and maps it to DocItemLabel.FORMULA 3.
• VLM prompts use <chem>...</chem> tags with reactive SMILES notation, and <chem> is an explicitly whitelisted tag in CHANDRA_ALLOWED_TAGS 4.
• The do_formula_enrichment pipeline option (default: False) enables VLM-based formula recognition via the CodeFormulaVlmModel 5. Enabling this processes DocItemLabel.FORMULA elements, not table-cell-embedded structures.

There are no chemical-notation-specific OCR models in Docling today; the <chem> handling is VLM-prompt-level post-processing, not a dedicated chemistry OCR layer.

Open Issues and In-Progress Work#

Several tracked issues and PRs are relevant to the long-term resolution of these limitations:

• docling-core #172 — Requests nesting support (inline formulas, images) in table cells and captions. A maintainer explicitly noted the need for image support inside table cells.
• PR: feat: enable rich tables 6 — Introduces RichTableCell type allowing table cells to hold images and other nested content. Merged 2025-08-27.
• PR: feat: add get_cell_image method to TableItem 7 — Adds an API to extract individual table cell images as crops. Open (as of 2026-07).
• PR #3107: fix: Use OCR cells with TableFormer v2 — Improves how OCR-derived cells are consumed in TableFormer V2, reducing some cell-matching failures. Merged 2026-03-11.
• keep_empty_clusters option (commit a436be7) — Adds a LayoutOptions.keep_empty_clusters flag (default False) that controls retention of empty clusters during layout postprocessing, which may help with empty-cell edge cases.

Workarounds#

Until rich table cell support and improved cell-image binding land fully:

• Use an external OCR service — AWS Textract correctly handles the pharmaceutical table cases that Docling misreads 1. Docling has an OCR plugin architecture; substituting a more capable backend may reduce errors.
• Image-based extraction — Export the page as a high-resolution image (generate_page_images=True, images_scale ≥ 2.0) and crop cells using their bounding-box provenance. This bypasses the text-cell binding step entirely.
• TableFormerMode.ACCURATE — Use table_structure_options.mode = TableFormerMode.ACCURATE with do_cell_matching=True 1; it is the best available mode even though it does not fully resolve the issue.
• Post-process with get_cell_image — Once PR #652 7 is merged, use the new TableItem.get_cell_image() method to retrieve visual crops of individual cells for downstream chemistry OCR tools (e.g., OSRA, MolScribe).

Key Source References#