dantesfai/TiDG-Corpus

Tigrinya Multiclass Text Classification Dataset Across a diverse Genre # Tigrinya Diverse Genre Corpus (TiDG) for Text Categorization The Tigrinya Diverse Genre Corpus (TiDG) is a clean, labeled dataset designed for multiclass text classification in Tigrinya, a low-resource Semitic language widely spoken in Eritrea and northern Ethiopia. The corpus comprises 8,067 text documents, covering seven diverse categories: Culture, Economy, Health, Politics, Religion, Science & Technology, and Sport. Each document is provided in both its cleaned Tigrinya script form and a SERA transliteration, supporting researchers who may prefer working with either script. The dataset is preprocessed, stratified, and split into training (80%), validation (10%), and testing (10%) subsets, ensuring robust evaluation of machine learning models. This dataset addresses the scarcity of benchmark corpora for Tigrinya and contributes to advancing research in natural language processing (NLP) for low-resource languages. Potential applications include document categorization, content recommendation, intent recognition, and domain-specific information retrieval. A label encoder is included to facilitate mapping between category names and numeric labels. The dataset is distributed in CSV format, accompanied by usage scripts and documentation. TiDG provides an essential resource for researchers aiming to build, evaluate, and benchmark text classification models for underrepresented languages, promoting inclusivity in multilingual NLP research. ## Dataset Description - **Language**: Tigrinya (with transliteration using the SERA standard) - **Total Samples**: 8,067 text documents - **Split**: - Train: 80% (6,453 samples) - Validation: 10% (807 samples) - Test: 10% (807 samples) - **Format**: CSV files (`train.csv`, `val.csv`, `test.csv`) - **Columns**: - `cleaned_text`: Preprocessed and normalized Tigrinya text - `transliteration`: SERA transliteration of the cleaned text - `label`: Numeric label corresponding to one of the categories ## Categories The dataset includes …