sudeep-07-hub/Multilingual-Text-Classification-Using-Shared-Semantic-Representations

This project investigates whether multilingual transformer models can classify text across multiple languages more effectively than traditional TF-IDF based approaches. The project selects four languages — English, Hindi, Tamil, and Swahili — representing different scripts, resource levels, and language families. # Multilingual Text Classification Using Shared Semantic Representations A comparative study of **TF-IDF**, **mBERT**, and **XLM-R** for multilingual topic classification on the **SIB-200** dataset across English, Hindi, Tamil, and Swahili. --- ## Research Question > Do shared semantic representations from multilingual transformers (mBERT, XLM-R) outperform non-shared TF-IDF features for multilingual topic classification across languages with different scripts and resource levels, and how does preprocessing strategy affect each representation type? ## Key Contributions | Layer | Contribution | Description | |---|---|---| | **Direction 1** | Classical vs Transformer | TF-IDF baseline vs mBERT/XLM-R shared representations | | **Gap 3** | Preprocessing Ablation | Aggressive vs minimal preprocessing — impact on each model type | | **Gap 5** | Language-wise Analysis | Per-language breakdown revealing hidden performance disparities | | **Gap 6** | Error Analysis | Qualitative misclassification analysis | ## Dataset: SIB-200 - **Source:** Davlan/sib200 on HuggingFace - **Task:** 7-class topic classification (science_technology, travel, politics, sports, health, entertainment, geography) - **Languages used:** English (`eng_Latn`), Hindi (`hin_Deva`), Tamil (`tam_Taml`), Swahili (`swh_Latn`) - **Samples:** 1,600 train + 400 test (400 train / 100 test per language) ## Models | Model | Representation | Classifier | |---|---|---| | TF-IDF + LR | Character n-gram TF-IDF (2-4 grams) | Logistic Regression | | TF-IDF + CNB | Character n-gram TF-IDF (2-4 grams) | Complement Naive Bayes | | mBERT + LR | `bert-base-multilingual-cased` [CLS] embeddings | Logistic Regression | | XLM-R + LR | `xlm-roberta-base` [CLS] embeddings | Logistic Regression | ## Preprocessing Pipelines - **Pipeline A (Aggressive):** Lowercase → remove numbers → remove punctuation → tokenize → remove English stopwords → remove single-char tokens - **Pipeline B (Minimal):** Strip whitespace → normalize …