chinmayjainnnn/LLMs-for-Translation-of-Low-Resource-Languages
Domain:
natural language processingRecord type:
project
Creator:
chi
Host:
Machine translation from assamese to english and vice versa using state of the art LLM's
# Hindi-English Machine Translation Using Large Language Models (LLMs)
Welcome to the repository for our research and development in Hindi-English machine translation leveraging Large Language Models (LLMs). This repository documents our experiments, results, and contributions toward creating robust translation models for Hindi and other Indic languages.
## 📑 **Overview**
This project focuses on benchmarking and fine-tuning open-source LLMs for Hindi-English translation. Once robust models are achieved, the approach will be extended to India's North-Eastern languages.
## **Key Contributions**
1. **Extensive Zero-Shot Benchmarking**:
- Evaluated state-of-the-art open-source LLMs: LLaMA, Mistral, Gemma, and OpenHathi.
2. **Exploration of Prompting Strategies**:
- Benchmarked using 0-shot, 2-shot, and 4-shot prompting.
3. **Fine-Tuning with QLoRA**:
- Implemented Parameter-Efficient Fine-Tuning (PEFT) with Low-Rank Adaptation (LoRA).
---
## **Models Evaluated**
1. **LLaMA 2**: Pretrained and fine-tuned LLMs by Meta AI (7B, 13B, and 70B parameters).
2. **Mistral**: 7B parameter model with advanced architectural features like grouped query attention.
3. **Gemma**: Transformer decoder-based LLM (2B and 7B parameters) by Google AI.
4. **OpenHathi**: Indic language-specific model optimized for Hindi-English translation.
---
## **Prompting Techniques**
### Zero-Shot Prompting
- No examples provided; model relies on its inherent understanding of the task.
### Few-Shot Prompting
- Provides task examples in the prompt, significantly improving performance.
**Best Prompt Template**:
```text
System: You are a Translator from English to Hindi:
User: What is the translation of the sentence given below?
[English]:
[Hindi]:
```
---
## **Fine-Tuning with QLoRA**
### Techniques
- **PEFT (Parameter-Efficient Fine-Tuning)**: Focus on task-relevant parameters to reduce computational cost.
- **LoRA (Low-Rank Adaptation)**: Adjusts layer relevance dynamically during fine-tunin …
Similar
Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource LanguagesLLM Probe: Evaluating LLMs for Low-Resource LanguagesToward Robust Multilingual Adaptation of LLMs for Low-Resource LanguagesMultilingual jailbreaking of LLMs using low-resource languagesFine-Tuning LLMs for Low-Resource Dialect Translation: The Case of LebaneseAmharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages