TSR Coordinate System Alignment#

The Core Mismatch#

RAGFlow's PDF parser uses two distinct coordinate spaces that must be reconciled before table reconstruction is possible:

The TSR model (TableStructureRecognizer, a subclass of Recognizer) 1 operates on cropped per-table images and returns rows, columns, headers, and spanning cells in the coordinate space of that crop. Text boxes, by contrast, are stored with Y coordinates shifted by a running sum of all preceding page heights — the page_cum_height array 2.

How page_cum_height Is Built#

During __images__() 3:

1. self.page_cum_height = [0] — initialized with a leading zero.
2. For each page image, img.size[1] / zoomin (the page height in PDF points) is appended 4.
3. After all pages are processed, the list is converted to a cumulative sum: self.page_cum_height = np.cumsum(self.page_cum_height) 5.

The result is an array where page_cum_height[i] is the total height of all pages before page i, making it a direct Y-offset lookup.

Pages are rendered at 72 × zoomin DPI (default zoomin=3, so 216 DPI) via pdfplumber 6, and all coordinate arithmetic is done in these scaled units.

Promoting Text Boxes to Cumulative Space#

After layout recognition (_layouts_rec), every text box's top and bottom are lifted into page-cumulative space 7:

boxes[i]["top"] += page_cum_height[page_number - 1]
boxes[i]["bottom"] += page_cum_height[page_number - 1]

This means all subsequent operations — including the overlap matching that links TSR structure to text content — work in the same unified Y space. 8

TSR Processing and Coordinate Bridging (_table_transformer_job)#

_table_transformer_job() is the central routine that:

1. Crops each table from its page image using layout bounding boxes (scaled by ZM) 9. The table's top and x0 at this stage are page-local, not cumulative.

2. Runs TSR on the cropped images: recos = self.tbl_det(imgs) 10. Output coordinates are relative to each crop's top-left corner.

3. Annotates each TSR item with its page number and table index but does not yet apply the cumulative offset 11. The raw rotated coordinates are preserved in x0_rotated, top_rotated, etc. 12 for later rotation correction.

4. Matches TSR structure to text boxes via overlap detection 13. This step requires both parties to share the same coordinate space — the page_cum_height offset is applied to self.boxes in _layouts_rec() before _table_transformer_job() is called, so the TSR structure boxes and the text boxes are directly comparable.

⚠️ Common pitfall: TSR components stored in self.tb_cpns retain image-local coordinates (with x0_rotated/top_rotated copies). Only the text boxes in self.boxes are in cumulative space. If you add new code that compares tb_cpns entries directly against boxes entries, you must account for this asymmetry.

The matching uses Recognizer.find_overlapped_with_threshold() (threshold 0.3) for rows, headers, and spanning cells, and find_horizontally_tightest_fit() for columns. Tags R, H, C, SP written onto each text box drive the downstream TableStructureRecognizer.construct_table() call.

Rotated Table Path: Three-Stage Transformation#

When auto_rotate=True (controlled by the TABLE_AUTO_ROTATE env var, 14), tables may be rotated before TSR. _ocr_rotated_tables() then maps coordinates back through three stages:

1. Rotated image → original image coords: _map_rotated_point() inverts the rotation transform (supports 0°/90°/180°/270°).

2. Original image coords → page-local coords: divide by ZM and add the table's x0/top offset from the layout box 15.

3. Page-local → page-cumulative: add self.page_cum_height[page_index] to every top/bottom 16.

The final box is inserted into self.boxes with full cumulative Y coordinates, replacing the original OCR boxes for that table region 17.

Key Files and Entry Points#