zuck30/swahili-llm-scratch

SwahiliGpt from Scratch # Swahili Swahiba LLM | SwahiliGpt from Scratch. This project presents a complete implementation of a language model built entirely from scratch, designed specifically for the mixed‑language communication style used across East Africa commonly referred to as **Kiswaenglish**. This form of speech naturally blends Kiswahili and English in daily conversation, a pattern rarely supported well by standard global language models. Unlike most existing solutions, this model was developed without relying on pre‑trained model weights. It combines **custom synthetic data** and **publicly available Swahili corpora** to reflect local language use, culture, and context. The entire system is optimized to run efficiently on standard consumer hardware, making it accessible to students, developers, and researchers without access to specialized infrastructure. Currently the core training and inference scripts are built for **Silicon macOS using MLX**, with support for **Windows and Linux** planned contributions from the community to extend compatibility are welcome. > **Please, Refer to this repo for the WebApp Interface of this project** > zuck30/SWAHIBA # Inspiration & Reference This work is inspired by the approach, teaching, endless tutorials and open contributions of **Andrej Karpathy** and **Alec Radford**, whose work has shown what is possible when building language models from first principles. The architecture follows the Transformer design introduced in the foundational paper: > **Attention Is All You Need** > papers.neurips.cc # How to Run Follow these steps in order from start to finish to set up, build, and run the model: # 1. Install Dependencies First, install all required libraries: ```bash pip install -r requirements.txt ``` > Note: For Windows and Linux, use the PyTorch‑compatible requirements when they become available. The equivalent versions using PyTorch (which runs on Windows and Lin …