alistermarc/cuatrollm-adaptive-low-resource-mt

# CuatroLLM: Niche Translations with Adaptive In-Context Learning This project explores the capabilities of CuatroLLM, a 1.3B parameter, four-language (English, French, Spanish, German) large language model, for niche machine translation tasks. We use an adaptive fine-tuning approach to enhance its performance on specialized domains and explore its potential for low-resource languages like Filipino. ## Base Model The experiments are centered around the `britllm/CuatroLLM` model, available on Hugging Face: huggingface.co ## Project Objectives This project has three primary objectives: 1. **Benchmark Replication:** Replicate the baseline performance of CuatroLLM on complex reasoning tasks (ARC-C, Hellaswag, PAWS-X, etc.) as reported in the original paper. 2. **Adaptive Fine-Tuning for EN-FR:** Evaluate and enhance CuatroLLM's English-to-French translation capabilities in the scientific domain using the adaptive machine translation (MT) approach, which leverages in-context learning from a few examples. 3. **Low-Resource Translation for EN-FIL:** Explore the feasibility of fine-tuning CuatroLLM for English-to-Filipino translation, a language it was not pre-trained on, using a small, compiled dataset. ## Repository Structure * `evaluation/`: Contains the Jupyter notebooks used to conduct the experiments for each objective. * `objective_1/`: Notebooks for replicating the baseline reasoning benchmarks. * `objective_2/`: Notebooks for fine-tuning and evaluating English-French translation. * `objective_3/`: Notebooks for the English-Filipino translation experiment. * `results/`: Contains the raw JSON output files from the evaluation scripts. ## Key Findings ### 1. Benchmark Replication We successfully replicated the complex reasoning benchmarks from the original CuatroLLM paper, confirming the baseline performance of the model across English, French, German, and Spanish. Our results were not statistically different from the r …