samadon1/dondo-nanos

On-device Twi/Ewe ASR: post-training compression of Khaya AI's DONDO (models, scripts, iOS example) # DONDO-nanos Post-training compression of Khaya AI's DONDO speech model for on-device ASR in Twi and Ewe. DONDO is the best open speech model I've found for these languages, but the multilingual version is a 605.8M-parameter Wav2Vec2-BERT Conformer, about 1.2 GB in half precision. That is too big for a mid-range phone, which is where it would get used. This repo compresses it and measures every step. - **Blog:** samadon1.github.io - **Models:** huggingface.co ## Results Blended error = 0.5·WER + 0.5·CER on WAXAL validation (Twi/Ewe mean), greedy CTC decode, normalized text. All rows are trained on the same WAXAL data (CTC, no distillation). | Model | Layers | Params | Disk (fp32) | CPU RTF | Blended | |---|---|---|---|---|---| | Fine-tuned parent | 24 | 605.8M | 2423 MB | 0.252 | 0.216 | | **Nano-L12** | 12 | 315.6M | 1262 MB | 0.132 | **0.261** | | Nano-L6 | 6 | 170.5M | 682 MB | 0.069 | 0.387 | | Nano-L3 | 3 | 98.0M | 392 MB | 0.036 | ~0.57 | Halving the model costs 0.045 blended error. int8 (ONNX Runtime) shrinks Nano-L12 to 319 MB (~4×) for almost no accuracy change. RTF is compute-seconds per audio-second; every model is already faster than real time on CPU, so the point of compression here is size, not speed. ## The recipe 1. Build a shallower Wav2Vec2-BERT student and **warm-start** it from evenly-spaced layers of the parent (24 → 6 takes layers 0, 5, 9, 14, 18, 23), copying the feature projection, adapter, and CTC head verbatim. 2. Fine-tune on WAXAL Twi + Ewe with a CTC loss. 3. Quantize to int8 with ONNX Runtime for deployment. Knowledge distillation from the parent was tested and dropped: the parent is out-of-domain on WAXAL, so its soft targets drag the student below plain fine-tuning. This is the same layer-initialization idea as DistilBERT and DistilHuBERT; the contribution here is the applied result for Ghanaian languages and the documented failure modes (naive int8 is a no-op on a Con …