VLM Coordinate System and Image Scaling#

DocTags-format VLMs (e.g., SmolDocling, Granite-Docling) encode spatial information as a sequence of four <loc_N> tokens per bounding box. Understanding how those tokens are generated, decoded, and mapped back to page coordinates is essential for debugging layout drift or configuring a new VLM preset.

The <loc_N> Grid (0–499)#

Each coordinate component is quantized to an integer in [0, 499] on a 500×500 normalized grid.

Encoding — DocumentToken.get_location_token() converts a single normalized float val ∈ [0, 1] to a token:

val_ = round(500 * val) # scale to grid
val_ = clamp(val_, 0, 499) # stay in range
→ <loc_{val_}>

A full bounding box is encoded as four consecutive tokens — x0, y0, x1, y1 — using DocumentToken.get_location(), which first normalizes pixel coordinates by page_w / page_h before quantizing.

Vocabulary — DocumentToken.get_special_tokens() generates <loc_0> through <loc_{max(xsize, ysize)-1}>, where both xsize and ysize default to 500. This is the vocabulary passed to LLM tokenizers 1.

Decoding Back to Page Coordinates#

load_from_doctags() (in docling-core/document.py) contains a nested helper, extract_bounding_box(), that reverses the encoding:

1. Extract integers via regex r"<loc_(\d+)>" — reads the first four matches as l, t, r, b 2.
2. Divide by 500 → normalized BoundingBox with coordinates in [0, 1].
3. Scale to page pixels via bbox.resize_by_scale(pg_width, pg_height), where pg_width / pg_height come from the PIL image passed alongside the tokens 3.

This means the final bounding box is in image-pixel space, not raw PDF-point space. The image dimensions are therefore the critical intermediary.

How VlmConvertOptions.scale Sets the Image Dimensions#

VlmConvertModel.__call__() requests a page image with 4:

image = page.get_image(scale=self.options.scale, max_size=self.options.max_size)

VlmConvertOptions.scale defaults to 2.0 and max_size defaults to None 5. A scale=2.0 doubles the pixel dimensions relative to the PDF's native point size.

Page.get_image() 6 applies a max_size cap if set:

if max_size:
    scale = min(scale, max_size / max(page_width, page_height))

The image that VlmConvertModel sends to the model inference engine is also the image passed to load_from_doctags. Therefore, when extract_bounding_box scales the normalized coordinates by pg_width / pg_height, it is scaling against the same scaled image the model saw. Bounding boxes will be in the scaled image's pixel space.

The _default_image_scale and the Two-Scale Distinction#

Page._default_image_scale is a separate concern from VlmConvertOptions.scale. It is set during page preprocessing by PagePreprocessingModel._populate_page_images():

page._default_image_scale = images_scale # from PdfPipelineOptions.images_scale
page.get_image(scale=images_scale) # warms the image cache

page.image (the property) returns the image at _default_image_scale 7. This scale is used by enrichment models and visualization utilities, not by VlmConvertModel. The VLM always calls page.get_image(scale=self.options.scale) directly, bypassing _default_image_scale.

Configuration Implications#

• Mismatched scale = wrong bounding boxes. If the image sent to the VLM differs in size from the image used to reconstruct bounding boxes, all spatial coordinates will be off. The DocTags path avoids this by using the same image for both model input and bbox decoding.
• max_size clips scale silently. When a max_size is set, Page.get_image() may silently reduce the effective scale 8. The actual pixel dimensions of the returned image determine the true coordinate space, not the nominal scale value.
• Image passed to DocTagsDocument must match. The PIL image in each (tokens_str, image) pair supplied to DocTagsDocument.from_doctags_and_image_pairs() must be the exact image the model processed. Using a differently-scaled version will shift all decoded bounding boxes proportionally.

Key Source Files#