TableFormer Confidence Scoring#

Overview#

TableFormer (both V1 and V2) internally computes token-level logits and bounding-box confidence signals during inference, but none of these scores are propagated through the pipeline's output interface into the DoclingDocument or TableData / TableCell types in docling-core. Downstream consumers have no direct access to per-table or per-cell reliability scores, so failures are currently silent.

The Gap: No Confidence Field in the Data Model#

TableCell and TableData in docling-core carry only structural information: cell text, span extents, bounding box, and semantic role flags (column_header, row_header, row_section) 1. There is no confidence field. The TableFormerV2Output class in docling-ibm-models does return logits and hidden_states 2, but the generate() method returns only generated_ids and predicted_bboxes 3 — scores are discarded before results leave the model layer.

The Docling pipeline adapters (TableStructureModel for V1, TableStructureModelV2 for V2) convert these raw outputs into Table objects and store them in TableStructurePrediction.table_map 4 5. Neither adapter extracts or forwards any confidence value from the model outputs.

BaseTableStructureModel defines only predict_tables() 6, with no provision for confidence scores in its contract.

Practical Impact: Silent Data Loss#

Because confidence is invisible, failures surface only as missing or wrong content — not as warnings or low-score flags.

Coverage failure is the most severe and measurable mode: when TableFormer predicts a grid that is smaller than the layout cluster's bounding box, PDF text cells outside every row/column band are silently dropped. Measured over a 24-PDF corpus, invoices lost 3.28% of words inside table bounding boxes, with worst-case documents losing 53% 7. The layout postprocessor had already transferred those text cells into the TABLE cluster, so they have no fallback route to body text 8.

Merged-cell mishandling (V1 and V2) is another silent failure: complex merged-row/column topologies may be reconstructed incorrectly across both model versions and both do_cell_matching settings, with no divergence signal emitted 9.

Proposed Indirect Quality Signals#

Since direct confidence propagation is not yet implemented, three indirect signals have been proposed for detecting unreliable table extraction:

1. Coverage Failure#

Compare the number of PDF text cells assigned to the layout cluster vs. cells emitted in table_cells. The probe is directly inspectable today:

cluster_cells = len(cluster.cells) # from page.predictions.layout
emitted_cells = len(table.table_cells) # from page.predictions.tablestructure

A large ratio (e.g., cluster had 30 cells, table emitted 2) is a direct indicator of grid under-prediction. PR #159 in docling-ibm-models marks fallback-matched cells with a negative confidence value — a partial signal that the matched cell position was inferred, not directly predicted 8.

2. Model Disagreement (V1 vs. V2)#

Running both V1 and V2 on the same table and comparing the resulting OTSL token sequences or num_rows/num_cols is a proposed uncertainty signal: when two models of the same architecture disagree on merged-cell topology or grid dimensions, the divergence itself flags low reliability. A structural_divergence flag was proposed to route such tables to a human-review queue instead of directly embedding them 9.

3. Layout Confidence#

The layout model emits a confidence field per cluster (e.g., cluster.confidence = 0.60). A low-confidence TABLE cluster that nonetheless triggers TableFormer processing is a leading indicator of a region that may not actually be a table 7. TABLE clusters at or near the 0.5 threshold (the per-label minimum for the postprocessor) are higher-risk candidates for misclassification.

Key Files#

Open Issues and Proposals#

• docling-ibm-models PR #159 — fixes row-band-match/no-column-band orphan case; the no-band-at-all case (grid far smaller than cluster) remains open; fallback matches are marked with negative confidence 8
• docling issue #3473 / PR #3753 — recover_orphaned_table_text option proposed to re-emit unabsorbed table-region text as body text 8
• docling-ibm-models issue #172 — follow-up to PR #159 tracking the no-band-at-all case with corpus measurements 7