IBM Watsonx and Granite Integration#

IBM's Granite model family and watsonx cloud service appear in Docling across two distinct contexts:

1. Local Granite Vision & Granite-Docling models — vision-language models (VLMs) shipped under ibm-granite/ on HuggingFace, used directly inside the Docling document-conversion pipeline for table structure, chart extraction, code/formula enrichment, and page conversion.
2. Watsonx as a remote LLM provider — Docling's knowledge-graph layer (docling-graph) routes structured-extraction requests to IBM's watsonx cloud service via LiteLLM.

Granite Models in the Docling Pipeline#

All Granite-backed pipeline stages require pip install docling[vlm] and download checkpoints from HuggingFace on first use. A CUDA GPU is strongly recommended; CPU inference is supported but slow. 1

Table Structure: Granite Vision#

GraniteVisionTableStructureOptions activates VLM-based table structure recognition using ibm-granite/granite-4.1-4b. The model receives a cropped table image with the <tables_otsl> prompt and returns an OTSL token sequence. Unlike the TableFormer backends, it does not run a cell-matching step — cell text comes directly from model output, so TableCell.bbox is always None for cells produced this way. 1

from docling.datamodel.pipeline_options import GraniteVisionTableStructureOptions, PdfPipelineOptions

opts = PdfPipelineOptions(do_table_structure=True)
opts.table_structure_options = GraniteVisionTableStructureOptions()

2

Chart Extraction: Granite Vision V3 and V4#

Chart data extraction (do_chart_extraction=True) uses Granite Vision models to convert bar, pie, and line charts into tabular data. Both variants extend BaseItemAndImageEnrichmentModel and only process PictureItems classified as bar_chart, pie_chart, or line_chart. Enabling chart extraction automatically enables picture classification. 3

3

from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.datamodel.chart_extraction_options import ChartExtractionModelOptions, ChartExtractionModelKind

opts = PdfPipelineOptions(
    do_chart_extraction=True,
    chart_extraction_options=ChartExtractionModelOptions(
        model=ChartExtractionModelKind.GRANITE_VISION_V4,
        chart2csv=True, # default
        chart2code=False,
        chart2summary=False,
    ),
)

3

Code/Formula Enrichment: Granite-Docling-258M#

The granite_docling preset selects ibm-granite/granite-docling-258M for the CodeFormulaVlmModel enrichment stage. It is one of two registered presets for code/formula extraction; the other is codeformulav2 (default). 4

from docling.datamodel.pipeline_options import CodeFormulaVlmOptions, PdfPipelineOptions

opts = PdfPipelineOptions(
    do_code_enrichment=True,
    do_formula_enrichment=True,
    code_formula_options=CodeFormulaVlmOptions.from_preset("granite_docling"),
)

The preset includes engine-specific overrides: MLX maps to ibm-granite/granite-docling-258M-mlx and Ollama maps to model name ibm/granite-docling:258m. Max new tokens is 8192 for this preset. 4

Full-Page VLM Conversion: Granite-Docling#

ibm-granite/granite-docling-258M (and its larger variants) can also be used as the VLM for whole-page document conversion via the vlm_pipeline_preset: "granite_docling" option in docling-serve's API 5 or via VlmPipelineOptions in the Python SDK. The preset can also be served via Ollama with model alias ibm/granite-docling:258m. 4

Watsonx as a Remote LLM Provider (docling-graph)#

In docling-graph, structured extraction from documents is delegated to an LLM via the LiteLLMClient. IBM watsonx is one of the natively registered providers alongside OpenAI, Mistral, Gemini, Bedrock, vLLM, and Ollama. 6

Configuration#

Watsonx requires both an API key and a project ID. The default base URL is https://us-south.ml.cloud.ibm.com but can be overridden via WATSONX_URL. The default model is ibm/granite-4-h-small. 7

6

The tokenizer for the watsonx provider is set to ibm-granite/granite-embedding-278m-multilingual (used for document chunking context estimates). 8

LiteLLM Model Name Resolution#

build_litellm_model_name() prefixes the model ID with watsonx/ when calling via LiteLLM (e.g., ibm/granite-4-h-small → watsonx/ibm/granite-4-h-small). Connection details—api_key, base_url, and project_id—are resolved from env variables and passed to LiteLLM's completion() call. 9

Watsonx-Specific Behavior#

The LiteLLMClient applies aggressive JSON cleaning specifically for the watsonx provider — the _needs_aggressive_cleaning() method returns True only when provider_id == "watsonx". This handles response formatting differences from the watsonx API when parsing structured extraction output. 10

Usage via PipelineConfig#

from docling_graph import PipelineConfig

config = PipelineConfig(
    source="document.pdf",
    template=MyExtractionTemplate,
    backend="llm",
    inference="remote",
    provider_override="watsonx",
    model_override="ibm/granite-4-h-small",
)
config.run()

11

Key Source Files#