ngusadeep/Gemma4-Swahili

Fine-tuning Gemma4 E2B (2B) and Gemma4 E24 (24B) Google's latest multimodal LLMs , specifically adapted for Swahili language tasks. # Gemma4 Swahili Fine-tuning (E2B & E24) Fine-tuning **Gemma4 E2B (2B)** and **Gemma4 E24 (24B)** for Swahili language instruction-following using LoRA + Unsloth. ## Models | Variant | Parameters | HuggingFace | | ------- | ---------- | ----------- | | **E2B** | ~2B | ngusadeep/gemma-4-2B-Swahili-llm | | **E24** | ~24B | ngusadeep/gemma-4-24B-Swahili-llm | ## Dataset Translated from `mlabonne/FineTome-100k` → **17,982 Swahili instruction-response pairs** via GPT-4o-mini Batch API. Dataset curated and published by **Samwel Ngusa**. | Format | Dataset | | ------ | ------- | | Alpaca | ngusadeep/Swahili-FineTome-Dataset | | ShareGPT | ngusadeep/Swahili-FineTome-Dataset-sharegpt | ## Training Configuration | Config | E2B | E24 | | ------ | --- | --- | | LoRA Rank | 64 | 128 | | Max Seq Length | 2048 | 2048 | | Batch Size | 4 | 2 | | Learning Rate | 5e-5 | 2e-5 | | Quantization | bfloat16 | 4-bit NF4 | | GPU | T4 16 GB / A100 40 GB | A100 40–80 GB | ## Inference ```python from unsloth import FastLanguageModel from unsloth.chat_templates import get_chat_template from transformers import TextStreamer model, tokenizer = FastLanguageModel.from_pretrained( model_name = "ngusadeep/gemma-4-2B-Swahili-llm", max_seq_length = 2048, load_in_4bit = False, ) tokenizer = get_chat_template(tokenizer, chat_template="gemma4") messages = [{"role": "user", "content": "Eleza nini maana ya uongozi."}] text = tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ).removeprefix(" ") model.generate( **tokenizer(text, return_tensors="pt").to("cuda"), max_new_tokens = 256, temperature = 1.0, top_p = 0.95, streamer = TextStreamer(tokenizer, skip_prompt=True), ) ``` ## Citation If you use these models or the dataset, please cite: ```bibtex @misc{gemma4_swahili_2026, author = {Samwel, Ngusa}, title = {Gemma4 Swahili: Fine-tuning Gemma4 E2B \& E24 for Swahili}, year = {2026}, publisher = {Hugging Face}, url …