tusharbhadana016/low-resource-doc-augmentation

# Low-Resource NLP Pipeline **Multilingual data augmentation via probabilistic backtranslation (22 languages), with BERT fine-tuning and LSTM baselines for document classification under data scarcity.** > This repository is a clean reconstruction of my 2023 university project, rebuilt from scratch since the original codebase was lost. The method and experimental design follow the original work, where probabilistic backtranslation delivered a **+4.2% accuracy uplift** over the no-augmentation baseline on low-resource benchmarks. ## Problem Text classifiers degrade sharply when labelled data is scarce — a common situation for specialised domains and most of the world's languages. This project tackles the problem from the data side: synthetically expanding a tiny training set with high-quality paraphrases, then measuring how much of the lost accuracy can be recovered with both a fine-tuned transformer (BERT) and a from-scratch recurrent baseline (BiLSTM + attention). ## Method ### Probabilistic backtranslation Each training example is round-trip translated (English → pivot → English) using MarianMT (Helsinki-NLP OPUS-MT) models across **22 pivot languages**: `ar bg cs da de el es et fi fr hi hu id it nl ru sk sv tr uk vi zh` Three sources of controlled randomness make the augmentation *probabilistic* rather than deterministic: 1. **Augmentation gate** — each example is paraphrased with probability `augment_prob` (Bernoulli), so training always mixes original and synthetic text. 2. **Pivot sampling** — the pivot language is drawn from a temperature-controlled categorical distribution. Low temperature concentrates on high-weight pivots; high temperature approaches uniform sampling over all 22. 3. **Stochastic decoding** — translation uses temperature sampling instead of pure beam search, so the same (text, pivot) pair can yield different paraphrases. Round trips are cached on disk, so repeated epochs and ablations never re-translate. ### Models | Model | D …