[3Xin4] Poster session 1

[3Xin4-19] Open Information Extraction from Document Images Using VQA Datasets

〇Atsuki Yamaguchi¹, Yasuhiro Sogawa¹ (1.Hitachi, Ltd.)

Keywords:Information Extraction, OpenIE, NLP

OpenIE is a popular paradigm to structure raw text data in a domain-agnostic way, extracting information as (subject; relation; object) triples. Despite much progress in text-based OpenIE, no previous work has so far proposed OpenIE methods for document images. This paper proposes a pipeline-based OpenIE system for document images, consisting of question generation, question answering, and triple extraction modules. The proposed system does not rely on annotated OpenIE datasets but only uses VQA datasets for training. Experimental results using DocVQA and InfographicVQA datasets demonstrate that the proposed method outperforms a text-based state-of-the-art OpenIE system: CompactIE in some cases. Further, our analysis also reveals the importance of employing the question answering module in the proposed method.