Anri-Lombard/sallm

Reproducible LLM training framework for South African languages. Contains code for MzansiLM (LREC 2026). # MzansiText & MzansiLM **An open corpus and decoder-only language model for South African languages.** This repository accompanies **"MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languages"** (arXiv:2603.20732). The public workflow is recipe-first: choose a recipe ID, inspect the Hydra configs it resolves to, then launch fine-tuning or evaluation. ```bash uv sync uv run sallm recipes list uv run sallm recipe show llama_t2x_xho uv run sallm finetune llama_t2x_xho --dry-run uv run sallm evaluate llama_t2x_xho --dry-run ``` Recipe IDs are defined in `recipes/registry.yaml`. Each recipe points to known Hydra configs under `src/conf`; those YAML files remain the source of truth for hyperparameters. ## Docs - Quickstart: install, inspect recipes, and dry-run the CLI. - Recipes: supported recipe IDs and their config targets. - Configuration: how recipe targets map to Hydra YAML. - SLURM: advanced cluster script compatibility notes. ## Paper Snapshot For exact reproduction of the LREC 2026 paper results, use the permanent snapshot: ```bash git clone github.com cd sallm git checkout tags/mzansitext-mzansilm-lrec2026-v1 ``` The `main` branch is actively maintained and may differ from the paper snapshot. ## Releases - Paper: arXiv:2603.20732 - Model: anrilombard/mzansilm-125m - Raw corpus: anrilombard/mzansi-text - Tokenized corpus: anrilombard/mzansi-text-tokenized - Collection: MzansiLM ## Repository Layout - `src/main/sallm`: Python package for training, fine-tuning, and evaluation. - `src/conf`: Hydra configs for model, data, fine-tuning, evaluation, and HPO. - `recipes/registry.yaml`: public recipe IDs mapped to known configs. - `ops/slurm`: advanced SLURM workflows for cluster execution. - `scripts`: local Python utilities. - `tokenizer`: tokenizer training and processing utilities. - `data`: dataset preparation and cleaning utilities. ## Development ```bash uv sync --extra d …