DeepDoc Module#

Overview#

DeepDoc is RAGFlow's document understanding subsystem, handling OCR, Document Layout Analysis (DLA), and Table Structure Recognition (TSR). It sits under deepdoc/ and can run either as an embedded library within the main RAGFlow process or as a standalone HTTP microservice.

The module has three functional layers:

Vision Models#

All vision models inherit from the Recognizer base class in deepdoc/vision/recognizer.py, which loads an ONNX session via load_model() and exposes input/output node names and shapes.

OCR#

The OCR pipeline (deepdoc/vision/ocr.py) uses two separate ONNX models: a text detector (det.onnx) and a text recognizer (rec.onnx). The top-level OCR class composes both, and when PARALLEL_DEVICES=N is set it instantiates one detector/recognizer pair per GPU for concurrent document processing 1.

Layout Recognizer (DLA)#

LayoutRecognizer classifies regions into 11 categories: _background_, Text, Title, Figure, Figure caption, Table, Table caption, Header, Footer, Reference, Equation. It checks DEEPDOC_URL / TENSORRT_DLA_SVR at init time; if set, inference is delegated to a remote DLAClient instead of the local ONNX session.

Table Structure Recognizer (TSR)#

TableStructureRecognizer produces 6 region types: table, column, row, column header, projected row header, spanning cell. Its output is in the coordinate space of the cropped table image, which must be reconciled against the page-cumulative Y offsets of OCR text boxes during PDF parsing 2.

ONNX Runtime & Hardware Support#

load_model() 3 is the central inference setup function:

1. Calls cuda_is_available() — imports torch and checks torch.cuda.is_available() for the target device ID 4.
2. If CUDA is available → creates ort.InferenceSession with CUDAExecutionProvider 5.
3. Otherwise → falls back to CPUExecutionProvider 6.

Key tuning env vars (all optional):

The pyproject.toml auto-selects the right package: onnxruntime-gpu on Linux/Windows x86_64, onnxruntime (CPU) on macOS or non-x86_64 1.

Huawei Ascend NPU: Layout and TSR support a parallel Ascend backend using Huawei's ais_bench library and compiled .om model files. Switch via LAYOUT_RECOGNIZER_TYPE=ascend or TABLE_STRUCTURE_RECOGNIZER_TYPE=ascend 1.

Deployment: Embedded vs. Standalone Microservice#

Embedded (default)#

The vision modules load directly in the RAGFlow main process. Models are fetched from rag/res/deepdoc/ or auto-downloaded from the InfiniFlow/deepdoc HuggingFace Hub repository.

Standalone Microservice#

The deepdoc/server/ directory implements a self-contained HTTP service:

• Framework: LiteServe 7
• Port: 9390 (default)
• Endpoints 8:
  • POST /predict/dla — Document Layout Analysis
  • POST /predict/ocr?operator=det|rec — Detection or recognition
  • POST /predict/tsr — Table Structure Recognition
  • GET /health — Health check

All prediction endpoints accept JPEG images via multipart/form-data with field name request. Responses are JSON bbox arrays.

The standalone image is built from Dockerfile_deepdoc_oss on Ubuntu 24.04 with Python 3.12, using only onnxruntime (CPU), and generating stub packages to avoid pulling in torch, pdfplumber, or Ascend dependencies 9. The server's minimal deps are declared in deepdoc/server/pyproject.toml: litserve, onnxruntime, opencv-python-headless, numpy, pillow, pyclipper, shapely.

Docker Compose 10: Activate with --profile deepdoc. The service uses the deepdoc_oss image and health-checks on GET /health every 10 seconds with up to 60 retries.

To point the main RAGFlow instance at the standalone service, set DEEPDOC_URL=http://<host>:9390 — LayoutRecognizer will route all DLA calls to the remote DLAClient instead of running them locally.

Document Parsers#

deepdoc/parser/ contains format-specific parsers: pdf_parser.py, docx_parser.py, excel_parser.py, html_parser.py, markdown_parser.py, epub_parser.py, ppt_parser.py, txt_parser.py, and several specialized variants (mineru_parser.py, docling_parser.py, paddleocr_parser.py, etc.). These call into the vision module and produce structured text boxes for downstream RAG chunking.

Key Files#