Kenny0bi/gbe

Research checkpoint to product: ONNX export with parity proof, INT8 quantization re-measured on the full test set, latency benchmarks, and a Yoruba diacritics demo that runs entirely in the browser # gbe Carrying my ami model from research checkpoint to product: ONNX export with a parity proof, INT8 quantization with the accuracy re-measured instead of assumed, the full latency story on a 2014 CPU, and a demo where the model runs in your browser with no server anywhere. **Try it: huggingface.co (mirrored at kenny0bi.github.io) *gbé* is Yoruba for "to carry". My ami project ended where most research projects end: a trained checkpoint, an evaluation table, a repo. Almost nobody types Yoruba on a machine with PyTorch installed. This project is the second half of the job: making the same model deployable, and proving nothing was lost on the way. ## The pipeline, with receipts **Export.** gbe/export.py pulls the published checkpoint from the Hub (gbe deliberately shares no code with the ami repo; the model snapshot is the single source of truth) and exports it to ONNX with dynamic batch and sequence axes. Then the proof: random batches through both graphs, max absolute logit difference 9.5e-6, and every one of the 6,573 MENYO-20k test sentences restores to the byte-identical string under torch and under ONNX. Export without a parity check is a rumor. **Quantize.** gbe/quantize.py applies dynamic INT8 quantization: weights stored as int8 with per-channel scales, activations quantized on the fly. 5.17 MB becomes 1.32 MB (3.92x). **Re-measure.** Quantization guides say "minimal accuracy loss". I re-ran ami's entire published evaluation for every backend rather than take anyone's word, mine included: | backend | char acc | word acc | ambiguous words | sentence exact | |---|---|---|---|---| | torch fp32 | 92.38% | 85.97% | 88.30% | 10.62% | | onnx fp32 | 92.38% | 85.97% | 88.30% | 10.62% | | onnx int8 | 92.39% | 85.99% | 88.32% | 10.63% | The int8 model comes out 30 net characters *ahead* out of 367,099 scored positions: quantization noise flipped a handful of coin-toss argmaxes, slightly more of them right than wrong. …