Parameter-Efficient Fine-Tuning for LLM-Based Arabic-to-English Machine Translation

Large Language Models (LLMs) such as GPT-3, BLOOM, BERT... have revolutionized natural language processing (NLP), particularly in translation. However, fine-tuning these models for downstream tasks, such as Arabic-to-English translation, requires extensive computational resources. Traditional full fine-tuning methods that involve updating all parameters of the model pose significant computational and memory challenges, notably for models with billions of parameters. This study investigates the application of LoRA-based PEFT methods on two chosen models for Arabic to English translation, AraT5 and NLLB-200, with a focus on understanding the trade-offs between computational efficiency and translation quality.