fgaim/TiQuAD

Tigrinya Question-Answering Benchmark Dataset # TiQuAD: Tigrinya Question Answering Dataset This repository accompanies our ACL 2023 paper **"Question-Answering in a Low-resourced Language: Benchmark Dataset and Models for Tigrinya"**. Selected for the Outstanding Paper Award. ## Overview Question-Answering (QA) has seen significant advances recently, achieving near human-level performance over some benchmarks. However, these advances focus on high-resourced languages such as English, while the task remains unexplored for most other languages, mainly due to the lack of annotated datasets. This work presents `TiQuAD`, the first human annotated QA dataset for Tigrinya, an East African language. The dataset contains 10.6K question-answer pairs (6.5K unique questions) spanning 572 paragraphs extracted from 290 news articles on various topics. The paper presents the dataset construction method, which is applicable to building similar resources for related languages. In addition to the gold-standard TiQuAD, we develop `Tigrinya-SQuAD`, a silver dataset used as additional training resource and created by machine translating and filtering the English SQuAD v1.1 dataset. We present comprehensive experiments and analyses of several resource-efficient approaches to QA, including monolingual, cross-lingual, and multilingual setups, along with comparisons against machine-translated silver data. Our strong baseline models reach 81% in the F1 score, while the estimated human performance is 92%, indicating that the benchmark presents a good challenge for future work. ## Datasets ### 1. TiQuAD v1 Human annotated question-answering dataset with entries. **📥 Download** via HuggingFace Hub | **Split** | **Articles** | **Paragraphs** | **Questions** | **Answers** | |-----------|--------------|----------------|---------------|-------------| | Train | 205 | 408 | 4,452 | 4,454 | | Dev | 43 | 76 | 934 | 2,805 | | Test* | 42 …