Abdisalam-Badel/SECSHSD

Somali-English Code-Switched Hate Speech Detection ## Requirements >📋 Requirements to install for LLM evaluation: ```setup pip install openai pip install sentence_transformers pip install rank_bm25 pip install faiss ``` >📋 Requirements to install for Fine-tuning BERT and RoBERTa: ``` pip install torch==2.5.1 (cuda==11.8) pip install transformers==4.46.3 pip install pandas==2.2.2 pip install scikit-learn== 1.4.2 ``` ## Training and Evaluation >📋 The hybrid framework: ```eval python hybrid.py ``` >📋 Evaluation: ```eval python evaluation.py ``` >📋 Baseline models including faiss and the fine-tuned models: ```train python bert_roberta.py python faiss.py ``` >📋 Exchange the model name, for BERT: bert-base-uncased and for RoBERTa: roberta-base. ## Data >📋 The hateful-words folder contains the hate lexicon, which includes: - hate.txt: Hate words introduced to the LLM with the classification prompt. >📋 The dataset folder contains: ``` Complete_dataset.json (has 3012 rows, and two columns: label and text), test_data.csv (has 1,200 rows with one column: text), test_data_ground_truth.csv (has 1,200 rows with two columns: label and text), and train.csv (has 1,812 rows with one column: text) ``` the Complete_dataset.json is the complete dataset. We put it here so that future users can divide the dataset as they want. ## Results Our hybrid model achieves the following performance across the various models: | Model name | Accuracy | Macro-F1 | | ------------------ |---------------- | -------------- | | gpt-3.5-turbo | 77.17 | 74.96 | | gpt-4o-mini | 74.33 | 73.97 | | gpt-4o | 66.58 | 66.34 | | deepseek-V3 | 78.33 | 77.51 | 📋 This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. ---> >📋 This repository is actively maintained by **Abdisalam** **Badel**. For any inquiries, please contact him at fiicane121@gmail.com.