Yinhan-Lu/many-shot-ICL4LRL

Codebase for the paper "An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages" # An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages **arXiv:** 2604.02596 **Authors:** Yinhan Lu, Gaganpreet Jhajj, Chen Zhang, Anietie Andy, David Ifeoluwa Adelani --- ## Quickstart Smallest end-to-end run — eng→Oro, 5-shot BM25 ICL with Gemini, on data from a public HuggingFace dataset (no token required): ```bash pip install -e . # 1. Data: Oro parallel corpus (public HF dataset howard-nlp/ibom-mt) python scripts/data/prepare_ibom_parallel_corpus.py --lang oro # 2. Prepare 5-shot BM25 inputs (997 requests) python -m manyshot_icl.pipeline.prepare_ibom_series \ --experiments bm25_random --languages oro --models gemini --shots 5 --no-chunking # 3. Inference (get a key at aistudio.google.com) export GOOGLE_API_KEY=... python -m manyshot_icl.pipeline.realtime_api \ --input-file experiments/bm25_random_ibom_997/batch_inputs/eng2oro/gemini_5shot.jsonl \ --output-dir experiments/bm25_random_ibom_997/batch_outputs/eng2oro/gemini_5shot \ --model-type gemini # 4. Evaluate (spBLEU / chrF++) python evaluation/evaluate_manyshot.py \ --batch-dir experiments/bm25_random_ibom_997/batch_outputs \ --data-dir data/parallel_corpus --skip-comet ``` The full experiment grids are driven by the per-experiment runners below. --- ## Repository structure ``` manyshot_icl/ # pipeline library: prompt construction, BM25/embedding # retrieval, request preparation, realtime inference client experiments/ # one runner per research question (prepare -> infer -> evaluate) # scaling/ retrieval/ domain/ ordering/ finetuning/ dictionary/ evaluation/ # spBLEU / chrF++ evaluators and result aggregation scripts/data/ # dataset acquisition and format converters configs/ # languages, models, and the paper's shot grid docs/DATA.md # per-dataset acquisition instructions ``` --- ## Experiments Each research question is driven by a dedicated runner script: | Experiment | Ru …