Arabic Facebook Corpus for Hate and Offensive Language Detection and Sentiment Analysis (MessBess)
Overview
MessBess is an Arabic Facebook corpus containing 391,510 comments prepared for research on hate and offensive language (HOF) detection and sentiment analysis.
The dataset was prepared as a fixed experimental resource to support reproducible evaluation of Arabic natural language processing models. It accompanies experiments based on MARBERT and several parameter-efficient fine-tuning (PEFT) approaches.
The released version includes the complete cleaned corpus together with the exact training, validation, and test partitions used in the associated experiments.
Dataset Composition
The final cleaned corpus contains:
391,510 Arabic Facebook comments
The dataset is distributed using fixed and mutually separated partitions:
Partition
Number of Comments
Training
274,057
Validation
58,726
Test
58,727
Total
391,510
These fixed partitions are released to facilitate direct reproducibility and fair comparison between models without requiring researchers to regenerate the experimental splits.
Annotation Tasks
The corpus supports two supervised text-classification tasks.
1. Hate and Offensive Language Detection
The first task is formulated as binary classification with two labels:
Clean: comments that do not contain hate or offensive language.
HOF: comments containing hate and/or offensive language.
2. Sentiment Classification
The second task is formulated as three-class sentiment classification with the following labels:
Positive
Neutral
Negative
Label Relationship
The sentiment annotation follows a constrained labeling scheme in which comments labeled as HOF are assigned to the Negative sentiment class.
Researchers should therefore consider this dependency when using the corpus for joint learning, multi-task learning, label-correlation analysis, or other experiments involving both tasks simultaneously.
Released Files
The Zenodo package contains the complete dataset and the fixed experimental partitions used in the associated study.
The archive includes:
clean_dual_label_corpus.csv
train.csv
validation.csv
test.csv
dataset_manifest.json
Complete Corpus
clean_dual_label_corpus.csv
Contains the complete cleaned dataset before separation into the fixed experimental partitions.
Training Partition
train.csv
Contains the 274,057 comments used for model training.
Validation Partition
validation.csv
Contains the 58,726 comments used for validation, model selection, and early stopping.
Test Partition
test.csv
Contains the 58,727 comments reserved for final model evaluation.
Dataset Manifest
dataset_manifest.json
Contains dataset-level information used to support integrity checking and reproducibility of the released experimental resource.
Reproducibility
The dataset is released with the exact train, validation, and test partitions used in the associated experiments.
The experimental pipeline performs multiple integrity checks before model training, including verification of:
dataset and partition sizes;
expected classification labels;
partition separation;
identifier overlap;
duplicate-text leakage across partitions;
class distributions; and
dataset file integrity.
The objective of releasing these fixed partitions is to allow other researchers to reproduce the experimental conditions without introducing variation caused by alternative random data splits.
Associated Experimental Code
The experimental implementation is maintained separately in the accompanying GitHub repository:
MARBERT-PEFT-Experiments
The repository contains the Jupyter Notebook implementing the experimental pipeline, including:
dataset verification;
tokenization;
MARBERT loading;
full fine-tuning;
frozen-encoder experiments;
parameter-efficient fine-tuning;
model training and validation;
test-set evaluation;
performance measurement;
computational-efficiency analysis; and
experimental result generation.
The GitHub repository and this Zenodo dataset are intended to be used together for reproduction of the associated experiments.
Experimental Context
The released dataset was used to evaluate Arabic transformer adaptation strategies based on MARBERT.
The associated experimental study investigates conventional and parameter-efficient adaptation approaches, including full fine-tuning and PEFT-based methods, under a common controlled evaluation protocol.
The primary predictive evaluation is based on Macro-F1, complemented by additional classification metrics and per-class evaluation.
Intended Use
MessBess is intended primarily for academic and research applications in areas including:
Arabic natural language processing;
Arabic social-media analysis;
hate speech detection;
offensive language detection;
sentiment analysis;
text classification;
transformer-based language modeling;
parameter-efficient fine-tuning;
multi-task learning;
representation learning; and
reproducible evaluation of Arabic NLP systems.
The fixed dataset partitions can also serve as a common experimental benchmark for comparing alternative classification architectures and adaptation strategies.
Citation and Version
Dataset version: v1.0
Researchers using this dataset are encouraged to cite the corresponding Zenodo record and, where applicable, the associated research article.
The Zenodo DOI provides the persistent identifier for this released version of the dataset.