aidyai/ibibio-nllb-ctranslate2

nllb-ctranslate2-converter for Ibibio # nllb-ctranslate2-converter Convert a fine-tuned NLLB (M2M100-architecture) Hugging Face checkpoint into CTranslate2 format, and optionally push the result straight to the Hugging Face Hub — all in a single script. CTranslate2 is a fast C++/CUDA inference engine for Transformer models. Converting your checkpoint to this format typically gives: - **2–5x faster inference** through layer fusion and padding removal - **Much lower memory usage**, especially combined with `int8` quantization - **Dynamic batching** support out of the box ## Requirements - Python 3.9+ - A Hugging Face NLLB checkpoint directory (containing `config.json`, `model.safetensors` or `pytorch_model.bin`, tokenizer files, etc.) - (Optional) a CUDA GPU if you want `float16` / `int8_float16` quantization ## Installation ```bash git clone github.com /nllb-ctranslate2-converter.git cd nllb-ctranslate2-converter pip install -r requirements.txt ``` ## Usage ### 1. Convert only ```bash python convert_and_push.py \ --checkpoint /content/checkpoint-27800 \ --output ./checkpoint-27800-ct2 \ --quantization int8_float16 ``` This produces a CTranslate2 model directory at `./checkpoint-27800-ct2`, including the tokenizer files copied over from the source checkpoint. > No GPU? Use `--quantization int8` instead — it runs entirely on CPU. ### 2. Convert, sanity-test, and push to the Hugging Face Hub ```bash python convert_and_push.py \ --checkpoint /content/checkpoint-27800 \ --output ./checkpoint-27800-ct2 \ --quantization int8_float16 \ --src-lang eng_Latn \ --tgt-lang ibo_Latn \ --test \ --push \ --repo-id your-username/nllb-checkpoint-27800-ct2 ``` This will: 1. Convert the checkpoint with `ct2-transformers-converter` 2. Copy tokenizer/config files into the output directory 3. Load the converted model and translate one test sentence, so you catch problems before uploading anything 4. Create (if needed) and push the model to `your-username/nllb-checkpoint-27800-ct2` on the Hugging Face Hu …