masakhane-io/africomet

COMET for African languages # AfriCOMET ### We have released the following AfriCOMET models to HuggingFace - AfriCOMET-STL-1.0 huggingface.co - AfriCOMET-MTL-1.0 huggingface.co - AfriCOMET-QE-STL-1.0 huggingface.co - AfriCOMET-STL-1.1 huggingface.co - AfriCOMET-QE-STL-1.1 huggingface.co # African Challenge Set - WMT 2024 Metrics Shared Tasks ## Overview The AfriMTE-ade-devtest-v2 dataset is a challenge set designed for evaluating machine translation quality across 13 African-centric language pairs. This dataset was used in the WMT 2024 Metrics Shared Task to support research and development in African machine translation evaluation and quality estimation. ## Language Pairs The dataset includes translations between the following language pairs: - Darija-French (ary-fra) - English-Egyptian Arabic (eng-arz) - English-French (eng-fra) - English-Hausa (eng-hau) - English-Igbo (eng-ibo) - English-Kikuyu (eng-kik) - English-Luo (eng-luo) - English-Somali (eng-som) - English-Swahili (eng-swh) - English-Twi (eng-twi) - English-isiXhosa (eng-xho) - English-Yoruba (eng-yor) - Yoruba-English (yor-eng) ## Dataset Structure The dataset contains four types of files for each language pair: 1. **Source Files** - Filename pattern: `AfriMTE-ade-devtest-v2.*.src.txt` - Contains source sentences 2. **Machine Translation Output Files** - Filename pattern: `AfriMTE-ade-devtest-v2.*.hyp-1.txt` - Contains machine-translated sentences 3. **Reference Translation Files** - Filename pattern: `AfriMTE-ade-devtest-v2.*.ref.txt` - Contains ground-truth reference translations 4. **Score Files** - Filename pattern: `AfriMTE-ade-devtest-v2.*.score.txt` - Contains human quality scores for machine translations ## Contact - Jiayi Wang (jiaywang@cs.ucl.ac.uk) - David Adelani (david.adelani@mila.quebec) ## Citations If you use this dataset i …