WilkieZA/Afrikaans_Next_Word_Prediction

BDatSci (Analytics and Optimisation) Research Project 2025 --- language: - af license: cc-by-nc-4.0 library_name: pytorch tags: - text-generation - next-word-prediction - afrikaans - gpt - transformer - autocomplete pipeline_tag: text-generation datasets: - custom metrics: - perplexity widget: - text: "Die weer is vandag" example_title: "Weather" - text: "Ek wil graag" example_title: "Desire" - text: "Die president het" example_title: "News" model-index: - name: Afrikaans-NWP-GPT results: - task: type: text-generation name: Next-Word Prediction metrics: - name: Keystroke Savings (KSS) type: keystroke_savings value: 0.0 # Update with actual value from evaluation - name: Mean Reciprocal Rank@3 (MRR@3) type: mrr value: 0.0 # Update with actual value from evaluation - name: Rank-Weighted Keystroke Savings@3 (RWKS@3) type: rwks value: 0.0 # Update with actual value from evaluation --- # Afrikaans Next-Word Prediction GPT **Author:** Louis Wilkinson (25948873) **Project:** BDatSci Research Project 2025 ## Model Description This model is a GPT-based next-word prediction system specifically designed for Afrikaans text. It uses a custom ByteLevel BPE tokeniser with a 12,000 token vocabulary and implements a token-aware prediction strategy for accurate next-word suggestions in Afrikaans language contexts. ## Model Architecture - **Type**: GPT (Generative Pre-trained Transformer) - **Layers**: 6 transformer blocks - **Attention Heads**: 8 - **Embedding Dimension**: 256 - **Context Window**: 32 tokens - **Vocabulary Size**: 12 000 tokens - **Tokeniser**: ByteLevel BPE with special tokens ### Special Tokens The tokeniser handles: - ` `: Personal names - ` `: Web addresses - ` `: Email addresses - ` `: Phone numbers - ` `: Numeric values - ` `: Emoji characters ### Afrikaans Contractions The model is trained to recognise common Afrikaans contractions: - `ek's`, `jy's`, `hy's`, `sy's`, `dit's`, `ons's`, `julle's`, `hulle's` - `daar's`, `hier's`, `wat's`, `dis`, `'n` ## Repository Structure ``` Afrikaans-NWP-GPT/ ├── model/ │ …