Chart Extraction#

Docling extracts chart data through two complementary mechanisms depending on the source format:

1. Native parser (openpyxl) — for Excel files, chart objects are read directly from the XLSX structure without any rendering step.
2. Vision Language Model (VLM) enrichment — for charts in PDF, PPTX, and image-based documents, a Granite Vision model converts the chart image to structured tabular data.

Both paths produce a PictureItem with a PictureMeta.tabular_chart field (TabularChartMetaField) carrying a TableData grid — giving downstream consumers a uniform data shape regardless of source format.

Excel / XLSX: Native openpyxl Path#

The MsExcelDocumentBackend reads XLSX files using openpyxl (requires pip install 'docling-slim[format-xlsx]'). Chart parsing is controlled by the parse_charts field on MsExcelBackendOptions — it defaults to True.

_find_chart_in_sheet() is called for both Worksheet and Chartsheet objects 1. For each chart in sheet._charts it:

• Looks up the DrawingML tagname in _CHART_TAGNAME_TO_CLASSIFICATION to set a PictureClassificationLabel (bar, line, pie, scatter, or OTHER_CHART).
• Calls _chart_to_table_data(), which resolves the chart's cell-range references back into the workbook (loaded with data_only=True so formulas are already computed) and reconstructs the underlying data as a TableData grid: one column per series, categories down the first column.
• Emits a PictureItem with PictureMeta carrying both the classification and the TabularChartMetaField.

Scatter charts use xVal/yVal references instead of cat/val 2. Chart titles are extracted from the DrawingML rich-text structure via _chart_title_text() and attached as a caption.

VLM Path: Granite Vision Enrichment#

For charts in PDF and other rendered documents, Docling uses a post-assembly enrichment model that runs after the main layout and OCR pass 3.

Enabling#

Chart extraction is off by default. Enable it via ConvertPipelineOptions 4:

from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.datamodel.chart_extraction_options import ChartExtractionModelOptions, ChartExtractionModelKind

pipeline_options = PdfPipelineOptions(
    do_chart_extraction=True,
    chart_extraction_options=ChartExtractionModelOptions(
        model=ChartExtractionModelKind.GRANITE_VISION_V4, # default
        chart2csv=True,
        chart2code=False,
        chart2summary=False,
    ),
)

Enabling do_chart_extraction automatically enables picture classification, which is required to identify chart types before inference 5.

Models#

Both model variants live in docling/models/stages/chart_extraction/granite_vision.py and extend BaseItemAndImageEnrichmentModel. They only process PictureItems whose picture classifier assigned a type in SUPPORTED_CHART_TYPES = ["bar_chart", "pie_chart", "line_chart"] 6. Elements not matching that predicate are passed through unchanged 7.

The V4 model uses torch.bfloat16 and optionally Flash Attention 2 when on CUDA 8. Models download automatically from HuggingFace if not present locally; pin to artifacts_path to avoid re-downloads.

Output modes (ChartExtractionModelOptions)#

Configured in docling/datamodel/chart_extraction_options.py — the default configuration enables only CSV output:

The VLM returns raw CSV text; the model parses it into a pandas DataFrame (inferring column headers from non-numeric first rows via _dataframe_to_tabledata()) then converts it to the same TableData shape used by the openpyxl path.

Device support#

Both model variants support CPU and CUDA; MPS is not listed 9.

Key Source Files#

Open / Planned#

A GitHub issue (#3771) tracks adding ChartExtractionApiOptions to route VLM inference to a remote OpenAI-compatible endpoint (vLLM, Ollama, TGI, LM Studio), mirroring the existing PictureDescriptionApiOptions pattern. The proposal also surfaces provenance concerns: remote endpoints may drift without a code change, so the design includes stamping resolved_model, endpoint_host, and reconstruction_source onto emitted PictureItems.