raypretam/low-res-nmt

Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Study # Two-step-low-res-NMT # Bilingual Text Translation and Evaluation Pipeline This repository provides a script to perform bilingual text translation and evaluation using a pre-trained multilingual language model, XLM-RoBERTa. The script includes steps for setup, BPE encoding, data binarization, translation generation, and evaluation. --- ## Table of Contents 1. Requirements 2. Setup Instructions 3. Pipeline Steps - Step 1: Environment Setup - Step 2: BPE Conversion - Step 3: Data Binarization - Step 4: Translation Generation - Step 5: Post-processing and Evaluation 4. Outputs --- ## Requirements Ensure the following are installed before running the script: - Python 3.6 - `pip` (Python package installer) - Required Python packages (installed in the script): - `sentencepiece` - `fairseq` - A pre-trained XLM-R model and its SentencePiece BPE model file. --- ## Setup Instructions 1. Clone this repository: `git clone ` `cd ` 2-stage Model : HI-KN Model XLM-R Model : XLM-R Download the 2-stage and XLM-R large models and place them in the repo. 2. Please paste the path of the XLM-R sentencepiece.bpe.model at line 23 of acl22-sixtp/fairseq/data/encoders/sentencepiece_bpe.py file. 3. Run `infer.sh`