Ascend Inference Pipeline#

The Ascend NPU path is a parallel inference stack that bypasses ONNX Runtime entirely, using Huawei's ais_bench library and compiled .om model files. It applies to layout recognition and table structure recognition (TSR) only — OCR text detection and recognition remain ONNX-based. 1

Backend Selection#

Two environment variables select the backend at runtime 2:

• Setting LAYOUT_RECOGNIZER_TYPE=ascend instantiates AscendLayoutRecognizer instead of the standard LayoutRecognizer.
• Setting TABLE_STRUCTURE_RECOGNIZER_TYPE=ascend routes TSR through _run_ascend_tsr() instead of the ONNX super().__call__.

Model Files#

Ascend requires Huawei-compiled .om files — standard .onnx files are incompatible. Models must be placed in rag/res/deepdoc/:

• Layout: {domain}.om (e.g., layout.om, layout.table.om) 3
• Table structure: tsr.om 4

Device selection uses the ASCEND_LAYOUT_RECOGNIZER_DEVICE_ID env var (default 0) for both pipelines. 5 6

Layout Pipeline: AscendLayoutRecognizer#

Initialization (lines 266–278)
Loads the .om file for the given domain and creates an InferSession. The session's input shape is read directly from the model: self.input_shape = self.session.get_inputs()[0].shape[2:4] (H, W). Unlike the ONNX backend, this class does not use the global loaded_models cache — the session is created per instance.

Preprocessing (lines 280–307)
Each image undergoes letterbox scaling: the image is resized to fit within the model's input shape while preserving aspect ratio, then padded symmetrically with gray (114, 114, 114) pixels. The preprocessing step stores the metadata needed to reverse the transform:

• scale_factor: [w/new_w, h/new_h] — scale ratios to map back to original pixel space
• pad: [dw, dh] — horizontal and vertical padding offsets added by letterboxing

Inference (lines 370–375)
The preprocessed image tensor is passed directly to session.infer(feeds=[ins["image"]], mode="static").

Postprocessing / Coordinate Restoration (lines 309–348)
For 6-column output [x1, y1, x2, y2, score, cls]:

1. Threshold raw detections by conf_thr.
2. Subtract pad offsets from the letterbox: xyxy[:,[0,2]] -= dw, xyxy[:,[1,3]] -= dh.
3. Multiply by scale_factor to restore original image coordinates.
4. Apply per-class NMS at IoU threshold 0.45.

TSR Pipeline: _run_ascend_tsr#

_run_ascend_tsr() creates a new InferSession per call (no persistent session). It reuses the parent class preprocess() (which also performs letterbox scaling) and postprocess() to produce detection results, then feeds those into the shared TableStructureRecognizer.__call__ logic for row/column alignment and span calculation. 7

OCR Box Association and Filtering#

After Ascend inference produces layout detections, the __call__ method performs the same OCR tagging pipeline as the ONNX backend:

1. Coordinate scaling: Layout bbox coordinates are divided by scale_factor (default 3) to convert from scaled-image space back to PDF page coordinates. 8

2. Deduplication: layouts_cleanup(bxs, lts) removes duplicate layout detections of the same type using bidirectional overlap matching. 9

3. OCR tagging via _tag_layout(ty) (lines 411–442): For each layout type (processed in priority order from footer → equation), each untagged OCR box is matched to a layout region using find_overlapped_with_threshold(box, lts_of_ty, thr=0.4). The bidirectional overlap threshold of 0.4 ensures neither box is a mere sliver of the other.

4. Garbage filtering: OCR boxes matching garbage patterns (bullets, page numbers, CID characters, bare URLs) are dropped inline. 10 Boxes for footer/header/reference layout regions are also dropped (unless position heuristics suggest they are falsely labeled), and any text appearing in 2+ dropped boxes is added to a global garbage set and removed at the end. 11 12

5. Placeholder injection: Figure/equation layout regions that were never matched to any OCR text box get a synthetic empty box appended so downstream rendering can still place them. 13

Key Differences from the ONNX Backend#

10 1

Related Articles#

• OCR Backend and Model Loading — multi-backend model loading/caching, including the global loaded_models cache that Ascend bypasses
• GPU and Accelerator Support — high-level ONNX Runtime provider architecture
• Layout Element Overlap Detection — overlapped_area, layouts_cleanup, and find_overlapped_with_threshold internals