NuExtract Transformers Compatibility#

Docling's structured extraction path (DocumentExtractor / ExtractionPipeline) relies on TransformersExtractionModel to run vision-language models (VLMs) locally via HuggingFace Transformers. The default model preset is numind/NuExtract-2.0-2B 1, loaded with AutoModelForImageTextToText and AutoProcessor.

This path has several known incompatibilities with specific Transformers versions that cause runtime errors. The models-vlm-inline extra in pyproject.toml explicitly excludes broken sub-ranges 2.

Breaking Changes by Transformers Version#

dtype kwarg removed for Qwen2-VL (Transformers 5.x)#

TransformersExtractionModel.__init__() calls AutoModelForImageTextToText.from_pretrained(..., dtype=...). Some Transformers 5.x builds dropped dtype as a valid kwarg for Qwen2VLForConditionalGeneration, producing:

TypeError: Qwen2VLForConditionalGeneration.__init__() got an unexpected keyword argument 'dtype'

This was a widely-reported regression 3 4. As of July 2026, the issue remains open with users reporting breakage on Transformers 5.12.1 4.

processor.tokenizer attribute missing (TokenizersBackend, Transformers 5.x)#

In Transformers 5.x, AutoProcessor.from_pretrained for pure-tokenizer processors returns a TokenizersBackend object that exposes _tokenizer instead of tokenizer. This breaks:

• build_nuextract_inputs() — calls processor.tokenizer.apply_chat_template(...)
• padding_side / pad_token access and StopStringCriteria in TransformersEngine

PR #3276 fixed the TransformersEngine path by introducing _get_tokenizer() (returns processor.tokenizer when present, falls back to processor itself), but the build_nuextract_inputs() call in prompt_utils.py still directly accesses processor.tokenizer 5.

AutoModelForVision2Seq removed (Transformers 5.x)#

PR #3200 migrated all model specs from the removed AUTOMODEL_VISION2SEQ type to AUTOMODEL_IMAGETEXTTOTEXT (AutoModelForImageTextToText). Any code or config still referencing Vision2Seq will fail on Transformers 5.x.

torch._dynamo import error#

A related failure (ImportError: cannot import name 'ConvertFrameReturn' from 'torch._dynamo.types') appears with certain Torch + Transformers 5.x combinations on macOS 3. The torch.compile() call applied to the NuExtract model at init time 6 is the trigger; it is guarded to skip on Python ≥ 3.14.

Role of qwen-vl-utils#

qwen-vl-utils is a hard runtime dependency for the NUEXTRACT prompt style. build_nuextract_inputs() raises an explicit ImportError if the package is absent. The functions used are:

• process_vision_info — extracts image tensors from Qwen-format message dicts
• fetch_image — fetches/decodes PIL images referenced in message content

The GRANITE_VISION prompt style does not use qwen-vl-utils; it calls processor.apply_chat_template() directly with a standard HF conversation format 7.

qwen-vl-utils>=0.0.11 is included in the models-vlm-inline extra 8. Install via:

pip install "docling[vlm]"
# or explicitly:
pip install "qwen-vl-utils>=0.0.11"

Current Version Constraints#

The models-vlm-inline extra in pyproject.toml pins:

The macOS upper bound is tighter (<5.9.0 vs <6.0.0); a comment in the file notes this is a "temporary solution until huggingface/transformers#46159 is resolved" 9. Transformers 5.0–5.3 are excluded on all platforms due to the Vision2Seq removal and TokenizersBackend regressions addressed by PR #3200 and PR #3276.

⚠️ Still unresolved: As of July 2026, users on Transformers 5.12.1 still report breakage 4, suggesting the upper-bound exclusions may not yet cover the full affected range.

Key Source Files#