Manthan-Hirani/Manthan_MISN

End-to-end Hindi–Marathi Neural Machine Translation project implementing both LSTM Seq2Seq with Bahdanau Attention and a modern Transformer-based architecture from scratch. Features SentencePiece BPE, RoPE, Grouped Query Attention (GQA), RMSNorm, BERT-style MLM, and GPT-style pretraining for low-resource machine translation. # Hindi-Marathi Neural Machine Translation This repository contains the final implementation for the MISN Lab, IIT Delhi - AdiVaani Initiative hiring assessment. It focuses on low-resource machine translation between Hindi and Marathi using both classical LSTM architectures and modern Transformer-based architectures with custom pre-trained representations. ## Repository Structure ```text . ├── src/ │ ├── Part1_Classical_NMT.ipynb # Complete LSTM Seq2Seq pipeline and experiments │ └── Part2_Language_Models.ipynb # Modern Transformer (RoPE, GQA, RMSNorm) & Pretraining ├── data/ │ ├── spm_bpe.model # SentencePiece BPE model │ └── spm_bpe.vocab # SentencePiece Vocabulary ├── plots/ # Output plots for Train/Val Loss, BLEU, and CHRF++ ├── experiments_and_rough_work/ # Exploration notebooks, debugging, and initial architecture tests ├── Technical_Report_Manthan.md # Detailed technical discussion, ablation studies, and analysis └── README.md # Reproducibility instructions ``` ## Dataset Setup Due to file size constraints, the massive parallel text corpus is not included in this repository. 1. Download the dataset provided for the assignment here: `shorturl.at` 2. Extract the contents. 3. Create a `data/processed/` directory in the root of this project and place the training and validation files (`train.hi`, `train.mr`, etc.) there so the Jupyter Notebooks can find them. ## Reproducibility Instructions ### Prerequisites Make sure you have a Python environment with PyTorch installed, ideally with CUDA support for GPU acceleration. ```bash pip install -r requirements.txt ``` ### Running Part 1: Classical NMT Open `src/Part1_Classical_NMT.ipynb`. - The notebook is designed to run end-to-end. It covers data loading, BPE tokenization, embedding initialization (both random and pre-trained BERT), the LSTM Encoder-Decoder training loop, and eval …