M$^{6}$Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis

• Main Authors: Cheng, Hiuyi, Zhang, Peirong, Wu, Sihang, Zhang, Jiaxin, Zhu, Qiyuan, Xie, Zecheng, Li, Jing, Ding, Kai, Jin, Lianwen
• Format: Journal Article
• Language: English
• Published: 15-05-2023
• Subjects: Computer Science - Computer Vision and Pattern Recognition
• DOI: 10.48550/arxiv.2305.08719

Document layout analysis is a crucial prerequisite for document
understanding, including document retrieval and conversion. Most public
datasets currently contain only PDF documents and lack realistic documents.
Models trained on these datasets may not generalize well to real-world
scenarios. Therefore, this paper introduces a large and diverse document layout
analysis dataset called $M^{6}Doc$. The $M^6$ designation represents six
properties: (1) Multi-Format (including scanned, photographed, and PDF
documents); (2) Multi-Type (such as scientific articles, textbooks, books, test
papers, magazines, newspapers, and notes); (3) Multi-Layout (rectangular,
Manhattan, non-Manhattan, and multi-column Manhattan); (4) Multi-Language
(Chinese and English); (5) Multi-Annotation Category (74 types of annotation
labels with 237,116 annotation instances in 9,080 manually annotated pages);
and (6) Modern documents. Additionally, we propose a transformer-based document
layout analysis method called TransDLANet, which leverages an adaptive element
matching mechanism that enables query embedding to better match ground truth to
improve recall, and constructs a segmentation branch for more precise document
image instance segmentation. We conduct a comprehensive evaluation of
$M^{6}Doc$ with various layout analysis methods and demonstrate its
effectiveness. TransDLANet achieves state-of-the-art performance on $M^{6}Doc$
with 64.5% mAP. The $M^{6}Doc$ dataset will be available at
https://github.com/HCIILAB/M6Doc.