shama-llama/hate-speech-detection

Adversarial and hierarchical transformer for Amharic hate speech detection # Hate Speech Detection for Amharic Language This project provides a pipeline for detecting hate speech in Amharic social media and online text. It combines multiple Amharic hate speech datasets, and then applies preprocessing and normalization. The detection system uses a state-of-the-art transformer model and applies adversarial and hierarchical architectures for classification. ## Architecture ## Data Sources The unified dataset is constructed from the following public Amharic hate speech datasets: - **SG2020** (Getachew, 2020): Dataset collected from Facebook pages of activists who write their posts using Geez script and comments of their followers. It is collected manually by going through each post and comment based on predefined rules. - **ZAK2021** (Zeleke, 2021): Extracted comments/posts pertaining to race, religion, and ethnicity using the Facepager API, resulting in a set of 30,000 comments between April 15, 2019 and December 15, 2019. A total of 5,000 comments/posts chosen at random for annotation. Three annotators (two candidate PhD. in Linguistics and one MSc. in Law) manually annotated the selected samples as “Hate” or “not-Hate” resulting 2,000 (1000 hate and 1000 non-hate) labeled comments. - **SM2022** (Minale, 2022): Dataset is prepared for hate speech detection and classification into four categories of speech. Namely, Normal speech, Racial Hate speech, Religious Hate speech, Gender Hate speech and Disability Hate speech. This dataset is collected from three social media sites: Facebook, Twitter, and YouTube. The collection is done automatically and the data is annotated by human annotators. The dataset is collected only for Amharic Language. - **MD2023** (Degu, 2023): Amharic text dataset extracted from memes in social media posts on Facebook and Telegram for hate speech detection / classification. - **RANLP2023** (Ayele et al., 2023): Collected using the Twitter API spanning from October 1, 2020 - November 30, 2022, considering t …