This is a duplication-free, high-quality, largest Swahili ONLY corpus. It is specifically designed for pre-training a compute-optimal language model of approximately 63 million parameters (following Chinchilla scaling laws for a 1.26B token corpus). All bit-for-bit identical content has been removed to ensure high training quality.
Unique Items: 979,304
Cleaned Tokens: ~1.262 Billion Tokens.