dadelani/AfriHG

News headline generation for African languages # AfriHG: News Headline Generation for African Languages paper | dataset ## Introduction We introduce AfriHG, a news headline generation dataset created by combining the XLSum and MasakhaNEWS datasets, focusing on 16 widely spoken African languages. We experimented with two sequence-to-sequence (seq2seq) models, mT5-base and AfriTeVa V2, as well as the Aya-101 large language model (LLM). Our results demonstrate that Africa-centric seq2seq models, such as AfriTeVa V2, outperform the massively multilingual mT5-base model. Additionally, we show that fine-tuning AfriTeVa V2 with 313M parameters yields competitive performance compared to prompting Aya-101 LLM, which has over 13B parameters. Using the Multilingual-Rouge metric provided accurate scores for Non-Latin Script Languages, addressing issues identified in the findings from AfriHG presented at AfricaNLP'24. ## Data Gathering We obtained the data by combinig the XL-Sum corpus with MasakhaNews corpus (of which most data are scraped from BBC news, and a few other websites listed in MasakhaNews. The full corpus can also be downloaded from AfriHG Google Drive Link ### Data Splits For all languages, we maintained the splits used in the XL-Sum and added all data from Masakhanews to the train data, while Languages like Shona (sna), Xhosa (xho), and Zulu (zul) that are not available in XL-Sum where splitted into a Train/Dev/Test split of 80/10/10 ratio used in the XL-Sum. The original splits were named `train`, `dev` and `test` and they correspond to the `train`, `validation` and `test` splits. Table below summarizes the Data Split and the Model Performance on the AfriHG with with the XL-Sum Abstractive Summarization Comparison results. | Language | Script | Train / Dev / Test | mT5-base (Summary) | mT5-base (Headline) | AfriTeVa V2-base (Headline) | Aya (Headline) | |----------------|--------|--------------------------|--------------------|---------------------|-----------------------------|----- …