Msingi-AI/Sauti

Kenyan Swahili Voice Model # sautiv0.1 Fine-tuning Design Document ## 1. Introduction This document outlines the design for fine-tuning a Swahili Text-to-Speech (TTS) model, named **sautiv0.1**, using the Google Waxal dataset. The fine-tuning process will leverage Modal for GPU-accelerated training and persistent storage. ## 2. Base TTS Model Selection: XTTS v2 ### Rationale XTTS v2 (from Coqui TTS) has been selected as the base model due to its robust performance, multilingual capabilities, and the availability of fine-tuning resources and community support. It is known for producing high-quality, natural-sounding speech and is well-suited for adaptation to new languages with sufficient data. ### Compatibility with Swahili While XTTS v2 has strong multilingual support, explicit confirmation of its performance on Swahili will be part of the evaluation phase. However, its architecture is designed to generalize across languages, making it a suitable candidate. ## 3. Fine-tuning Architecture Overview Fine-tuning XTTS v2 typically involves adapting the pre-trained model's weights to a new dataset. The core components of the XTTS v2 architecture include: - **Encoder**: Processes the input text into a sequence of hidden representations. - **Decoder**: Generates mel-spectrograms from the encoder's output. - **Vocoder**: Converts mel-spectrograms into raw audio waveforms. During fine-tuning, the entire model or specific layers (for example, language-specific layers in the encoder) are updated using the Swahili data. The goal is to retain the general speech synthesis capabilities of the pre-trained model while learning the specific phonetic and prosodic characteristics of Swahili from the Waxal dataset. ## 4. Dataset Processing Pipeline ### 4.1 Dataset Source: Google WaxalNLP (`swa_tts` subset) The `google/WaxalNLP` dataset on Hugging Face provides a dedicated `swa_tts` subset for Swahili Text-to-Speech. This dataset contains: - `id`: Unique identifier. - `speaker_id`: Unique identifier …