dsfsi/PuoBERTa

A Roberta-based language model specially designed for Setswana, using the new PuoData dataset. # PuoBERTa: A Curated Setswana Language Model A RoBERTa-based language model specially designed for Setswana, trained on the PuoData dataset for accurate and culturally relevant NLP applications. **Try it now:** Interactive Demo | Model on HuggingFace | Paper Give Feedback 📑: DSFSI Resource Feedback Form --- ## Table of Contents - Quick Start - Model Details - Installation - Usage Examples - Downstream Performance - Pre-Training Dataset - Citation - Contributing - Contact --- ## Quick Start ### Try Online (No Installation Required) Visit our **Interactive Demo** to try all PuoBERTa models in your browser: - **Fill-Mask**: Predict masked words in Setswana text - **News Classification**: Categorize Setswana news articles - **Named Entity Recognition (NER)**: Extract entities from text - **Part-of-Speech (POS) Tagging**: Identify grammatical roles of words ### Quick Start with Code Get started with PuoBERTa in just a few lines of code: ```python from transformers import pipeline # Use the fill-mask pipeline fill_mask = pipeline('fill-mask', model='dsfsi/PuoBERTa') result = fill_mask("Setswana ke puo ya .") print(result) ``` For more detailed examples, check out the examples directory with ready-to-run scripts for various use cases. --- ## Model Details ### Model Description This is a masked language model trained on Setswana corpora, making it a valuable tool for a range of downstream applications from translation to content creation. It's powered by the PuoData dataset to ensure accuracy and cultural relevance. - **Developed by:** Vukosi Marivate (@vukosi), Moseli Mots'Oehli (@MoseliMotsoehli) , Valencia Wagner, Richard Lastrucci and Isheanesu Dzingirai - **Model type:** RoBERTa Model - **Language(s) (NLP):** Setswana (BCP-47: `tn`) - **License:** CC BY 4.0 - **Training Dataset:** PuoData --- ## Installation Install the required dependencies: ```bash pip install transformers torch ``` For fine-tuning and advanced usage: ```bash pip install …