Bawilla/Njo-Viri

Balanda language training data # Njo-Viri — Balanda Language Dataset & Fine-tuning A parallel corpus for **Balanda (Njo Viri)**, a Bantu language, with tools to fine-tune a neural machine-translation model (English ↔ Balanda) on Kaggle. --- ## Dataset statistics | Split | Records | |-------|---------| | train | 3 669 | | val | 431 | | test | 217 | | **total** | **4 317** | Records are deduplicated, Unicode-normalised (ŋ ï ü ö), and validated. Each record: `{"instruction": "...", "input": " ", "output": " "}`. --- ## Repository layout ``` Njo-Viri/ ├── prepare_dataset.py # Step 1 – clean & split the corpus ├── balanda_finetune.ipynb # Step 2 – Kaggle fine-tuning notebook ├── train.jsonl # 85 % split (generated) ├── val.jsonl # 10 % split (generated) ├── test.jsonl # 5 % split (generated) ├── / # Raw JSONL source files └── README.md ``` --- ## Quick start ### Step 1 — Regenerate train/val/test splits (optional) Requires Python 3.10+ with no extra packages. ```bash git clone github.com cd Njo-Viri python prepare_dataset.py ``` This will print extraction counts per file and write `train.jsonl`, `val.jsonl`, `test.jsonl` into the repo root. --- ### Step 2 — Fine-tune on Kaggle #### 2a. Create a Kaggle Dataset 1. Go to kaggle.com → **New Dataset** 2. Name it exactly `balanda-dataset` 3. Upload `train.jsonl`, `val.jsonl`, `test.jsonl` 4. Click **Create** #### 2b. Add your HuggingFace token as a Kaggle Secret 1. Open any Kaggle notebook → **Add-ons → Secrets** 2. Add a secret named `HF_TOKEN` with your HuggingFace write token (get one at huggingface.co) #### 2c. Create and run the notebook 1. Upload `balanda_finetune.ipynb` to Kaggle 2. Attach the `balanda-dataset` dataset (Add data → Your datasets) 3. Enable **GPU T4 x2** accelerator 4. In **Cell 2 (Config)**, set `hf_repo_id` to `YOUR_HF_USERNAME/nllb-balanda-lora` 5. **Run All** The notebook will: - Install al …