Multilingual Cyber Threat Intelligence Vendor Classification Using Multilingual BERT (mBERT): An Empirical Analysis of Class Imbalance Mitigation Strategies

When a critical zero-day vulnerability is disclosed in a French-language security advisory or a Hausa-language threat bulletin, most deployed vendor identification systems simply fail, not because the vulnerability is obscure, but because the intelligence pipeline was never designed to read anything but English. We present an empirical evaluation of a multilingual vendor identification system for Cyber Threat Intelligence (CTI) that applies contextualised embeddings from Multilingual BERT (mBERT, bert-base-multilingual-cased) to classify vulnerability text across five languages; English, French, Hausa, Yoruba, and Igbo into 174 distinct vendor categories drawn from the NIST National Vulnerability Database (NVD). The AI contribution is a systematic controlled ablation of three class imbalance mitigation strategies; Synthetic Minority Over-sampling Technique (SMOTE), weighted cross-entropy loss, and their combination applied to a custom ThreatClassifier neural network head trained on frozen mBERT embeddings. The engineering application is automated vendor attribution in operational CTI pipelines, where accurate and fast vendor identification directly governs patch prioritisation . Results show that SMOTE oversampling improves macro F1 by 5.3% over the unmitigated baseline (0.8938 vs. 0.8409), while weighted loss alone reduces overall accuracy by 32% under extreme 174-class imbalance, a finding that contradicts common practice. The hybrid strategy combining SMOTE with weighted loss produces results statistically identical to SMOTE alone, exposing a fundamental redundancy: SMOTE’s rebalancing of the training distribution neutralises the corrective mechanism that weighted loss was designed to provide. Precision-Recall Average Precision reaches 0.977 (micro) and 0.979 (macro) for the best model, against a random baseline