stccenter/Comparative-Analysis-of-BERT-and-GPT-for-Classifying-Crisis-News-with-Sudan-Conflict-as-an-Example

Comparative Analysis of BERT and GPT for Conflict-Related Multiclass Label Classification from Multi-Label Ground Truth # Comparative Analysis of BERT and GPT for Classifying Crisis News with Sudan Conflict as an Example 🚀 📄 Paper     ## Updates ## Table of Contents 📌 - Overview 🔍 - Key Results 📊 - Model Zoo 🏛️ - Quick Start 🚀 - License 📜 - Citation 🔖 ## Overview This project compares Multi-Label Classification (MLC) and Single-Label Classification from Multi-Label Ground Truth (SL-MLG) for conflict-related news using BERT and GPT-based large language models (LLMs). The evaluation includes: - Fine-tuned BERT models (standard and hyperparameter-tuned) - GPT models using Zero-shot, Retrieval-Augmented Generation (RAG), and RAG with In-Context Learning (ICL) ## Key Results ### **MLC Collective Experimental Results** | Algorithm | Model | Runtime | F1-score | |-----------------|-------------------|-----------|------------| | **RAG ICL** | Llama3.1-70b | 3m 30s | 67.00% | | **RAG** | Llama3.1-70b | 2m 30s | 68.80% | | **Zero-Shot** | Llama3.3-70b | 1m 31s | 62.75% | | **BERT (tuned)**| bert-base-uncased | 187m 45s | 63.01% | | **BERT (tuned)**| bert-large-uncased| 1350m 55s | **68.90%** | | **BERT** | bert-base-uncased | 9m 43s | 62.85% | | **BERT** | bert-large-uncased| 14m 55s | 54.62% | ### **SL-MLG Collective Experimental Results** | Algorithm | Model | Runtime | Accuracy | |-----------------|-------------------|-----------|------------| | **RAG ICL** | Llama3.3-70b | 3m 12s | **90.70%** | | **RAG ICL** | Llama3.1-70b | 3m 14s | **90.70%** | | **RAG** | Llama3.3-70b | 1m 44s | 86.05% | | **Zero-Shot** | Llama3.3-70b | 40s | 76.74% | | **BERT (tuned)**| bert-base-uncased | 145m 18s | 74.42% | | **BERT (tuned)**| bert-large-uncased| 1243m 28s | 83.72% | | **BERT** | bert-base-uncased | 10m 5s | 76.74% | | **BERT** | bert-large-uncased| 28m 31s | 7 …