Information Extraction from Multi-Layout Invoice Images using FATURA Dataset

Abstract Document analysis and understanding models often require extensive annotated data to be trained. However, various document-related tasks extend beyond mere text transcription, requiring both textual content and precise bounding-box annotations to identify different document elements. Collecting such data becomes particularly challenging, especially in the context of invoices, where privacy concerns add an additional layer of complexity. In this paper, we introduce FATURA, a pivotal resource for researchers in the field of document analysis and understanding. FATURA is a highly diverse dataset featuring multi-layout, annotated invoice document images. Comprising $10,000$ invoices with $50$ distinct layouts, it represents the largest openly accessible image dataset of invoice documents known to date. We also provide an extensive evaluation of different information extraction methods under diverse training and evaluation scenarios, including a visual-based approach using object detection for text region classification, a multi-modal strategy integrating visual and textual data for granular content comprehension, and finally, a hybrid approach combining these methods. The dataset is freely accessible at this \href{zenodo.org, empowering researchers to advance the field of document analysis and understanding.