mewaeltsegay/Tigrinya_QA_training

All the different experiments done to train a dedicated QA model for Tigrinya. # Desta 1B QA Fine-tuning Project This repository contains code and experiments for fine-tuning the Desta 1B language model on question answering (QA) tasks. ## Project Overview The project focuses on: - Fine-tuning the Desta 1B model for QA applications - Data augmentation techniques for training - Multiple model versions with experimental improvements - Performance evaluation and error analysis ## Directory Structure - `desta_1b_QA_*/` - Different versions of fine-tuned models - `desta_1b_QA_finetuned/` and `desta_1b_QA_finetuned_v2/` - LoRA adapter versions - `dataset/` - Training and evaluation datasets - `training_history/` - Training logs and history - `tokenizer/` and `tokenizer_expanded/` - Tokenizer configurations ## Notebooks - `train_v*.ipynb` - Different versions/experiments - `train_span_extraction.ipynb` - Span extraction fine-tuning - `anlysis.ipynb` - Analysis and evaluation - `augmentation_test.json` - Data augmentation tests ## Requirements See `requirements.txt` for dependencies. ```bash pip install -r requirements.txt ``` ## Usage Run the training notebooks to fine-tune the model: ```bash jupyter notebook train.ipynb ``` ## Models Multiple model versions are available, each with different training configurations and improvements: - v3, v4, v5, v6, v7, v10, v102 - Full fine-tuned versions - finetuned, finetuned_v2 - LoRA adapter versions Each model directory contains: - `adapter_config.json` / `config.json` - Model configuration - `adapter_model.safetensors` / `model.safetensors` - Model weights - `tokenizer.model` - Tokenizer - `training_summary.json` - Training metrics - `eval_results.json` - Evaluation results