castorini/africlirmatrix

AfriCLIRMatrix is a test collection for cross-lingual information retrieval research in 15 diverse African languages. # AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages AfriCLIRMatrix is a test collection for cross-lingual information retrieval research in 15 diverse African languages. This resource comprises English queries with query–document relevance judgments in 15 African languages automatically mined from Wikipedia ## Paper AfriCLIRMatrix: Enabling Cross-Lingual Information Retrieval for African Languages ## Download Corpus ### Version 1.1 #### 1. Dataset (topic, qrels, folds, collections)      Afr \| Amh \| Ary \| Arz \| Hau \| Ibo \| Nso \| Sna \| Som \| Swa \| Tir \| Twi \| Wol \| Yor \| Zul     The dataset (v1.1) is also available on HuggingFace Dataset: - castorini/africlirmatrix - castorini/africlirmatrix-qrels-topics #### 2. Pre-built sparse index (for BM25)      Afr \| Amh \| Ary \| Arz \| Hau \| Ibo \| Nso \| Sna \| Som \| Swa \| Tir \| Twi \| Wol \| Yor \| Zul #### 3. Pre-built dense index (for mDPR)      Afr \| Amh \| Ary \| Arz \| Hau \| Ibo \| Nso \| Sna \| Som \| Swa \| Tir \| Twi \| Wol \| Yor \| Zul #### 4. Checkpoints - castorini/mdpr-tied-pft-msmarco ## Baseline Results Baseline BM25, mDPR (fine-tuned on ms marco) and sparse-dense hybrid results on Africlirmatrix ### nDCG@10 | | Afr | Amh | Ary | Arz | Hau | Ibo | Nso | Sna | Som | Swa | Tir | Twi | Wol | Yor | Zul | avg | |----------------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:| | BM25 (default) | 0.434 | 0.159 | 0.167 | 0.268 | 0.508 | 0.518 | 0.445 | 0.262 | 0.305 | 0.418 | 0.080 | 0.513 | 0.134 | 0.484 | 0.247 | 0.329 | | mDPR (MS Marco) | 0.309 | 0.215 | 0.355 | 0.118 | 0.269 | 0.338 | 0.282 | 0.351 | 0.218 | 0.335 | 0.265 | 0.333 | 0.232 | 0.377 | 0.178 | 0.281 | | Hybrid | 0.464 | 0.228 | 0.350 | 0.257 | 0.508 | 0.580 | 0.526 | …