InstituteforDiseaseModeling/Bridging-the-Gap-Low-Resource-African-Languages

Code and data corresponding to paper Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments # Bridging the Gap: Enhancing LLM Performance for Low-Resource African Languages with New Benchmarks, Fine-Tuning, and Cultural Adjustments **Authors:** **Tuka Alhanai** , **Adam Kasumovic** , **Mohammad Ghassemi** , **Aven Zitzelberger** , **Jessica Lundin** , **Guillaume Chabot-Couture** This repository contains the benchmarks, results, and all the code required to reproduce the results, tables, and figures presented in our paper. More specifically, this repository contains: 1. **Translated Winogrande Benchmarks:** Human and machine translations of Winogrande into 8 African languages: Shona, Igbo, Bambara, Amharic, Sepedi, Sesotho, Swtswana, and Tsonga (as well as preexisting translations into Afrikaans, Zulu, and Xhosa). 2. **Translated MMLU-Clinical Benchmarks:** Human and machine translations of the clinical sections "college medicine", "clinical knowledge", and "virology" of MMLU into 8 African languages: Shona, Igbo, Bambara, Amharic, Sepedi, Sesotho, Swtswana, and Tsonga (as well as preexisting translations into Afrikaans, Zulu, and Xhosa for the "clinical knowledge" and "college medicine" sections; we translated the "virology" section into Afrikaans, Zulu, and Xhosa as well in this release). 3. **Human Annotation of Winogrande:** Human annotations of the Winogrande dataset assessing translation quality and appropriateness in 11 African languages: Shona, Igbo, Bambara, Amharic, Sepedi, Sesotho, Swtswana, Tsonga, Afrikaans, Zulu, and Xhosa. 4. **Scripts to Reproduce Results:** Code used to regenerate the results, tables, and figures presented in the paper. Note that Meta's Belebele in English as well as Shona, Igbo, Bambara, Amharic, Sepedi, Sesotho, Swtswana, Tsonga, Afrikaans, Zulu, and Xhosa is also included since it was used as an African language evaluation benchmark for our experiments. A PDF-version of a slideshow presentation depicting our work is also provided in this repository. Our translated datasets can also be accessed on HuggingFace …