Pysham0n/Tunisian-Dialect-Abusive-Language-Detection

Abusive language detector in comments written in the Tunisian dialect. Using a combination of web scraping, deep learning, and state-of-the-art NLP models. # Tunisian Dialect Abusive Language Detection This project focuses on detecting abusive language in comments written in the Tunisian dialect. Using a combination of web scraping, deep learning, and state-of-the-art NLP models, the system identifies abusive content in Tunisian comments from YouTube. --- ## 🚀 Features - **Web Scraping**: Scraped Tunisian comments from YouTube using the YouTube Data API. - **Deep Learning**: Leveraged CNN-LSTM architecture for abusive language classification. - **Transformer Models**: Used pre-trained BERT models fine-tuned for the Tunisian dialect. - **Tunisian Dialect Focus**: Tailored for comments in Tunisian Arabic dialect. --- ## 🔧 Installation 1. Clone this repository: ```bash git clone github.com cd Tunisian-Dialect-Abusive-Language-Detection ``` 2. Create a virtual environment and activate it: ```bash python -m venv env source env/bin/activate ``` 3. Install the required dependencies: ```bash pip install -r requirements.txt ``` 4. Set up your API key for YouTube Data API in the scrapper script: ```bash YOUTUBE_API_KEY=your_api_key ``` ## 🗂️ Dataset - Comments were scraped from YouTube using the YouTube Data API. - The dataset was preprocessed to clean text, remove noise, and label abusive and non-abusive comments. - Stored in the `Dataset/` folder. --- ## 💻 Models Used ### CNN-LSTM - Combines the power of **Convolutional Neural Networks (CNNs)** for feature extraction and **Long Short-Term Memory (LSTM)** networks for sequential modeling. - Implemented in **TensorFlow/Keras**. ### BERT (Bidirectional Encoder Representations from Transformers) - Fine-tuned on the preprocessed Tunisian comments dataset. - Enabled state-of-the-art performance in abusive language detection. --- ## 🧪 Training and Evaluation ### Training - Train the models using `train_model.py`: ```bash python train_model.py --model cnn-lstm ## 📈 Example Usage - Run the detection scrip …