ArmelRandy/ICL-MT

[NAACL 2025 Findings] Example Selection via Similarity Search improves Low-resource Machine Translation # Official Repo of Example Selection via Similarity Search Official implementation of the paper Example Selection via Similarity Search improves Low-resource Machine Translation with code, scripts and outputs. ## Setup The requirements include PyTorch which should be installed in accordance with the type of GPU available on the device. Following its installation, we recommend to clone the directory in order to use it. ``` git clone github.com cd ICL-MT pip install -r requirements.txt ``` ## Overview In this paper, we study the impact of carefully chosen in-context demonstrations on the quality of MT outputs. We work with decoder-based LLMs and we study multiple similarity metrics and their impact on the translation task from English to other languages. Given a pool $\mathcal{P} = \{(x_i, y_i) : 1 \leq i \leq |\mathcal{P}| \}$ and a sentence $x$ to translate in $k$-shot, we find $i_1, \ldots, i_k$ such that $\sum_{1 \leq j \leq k}sim(x_{i_j}, x)$ is maximum and use the corresponding pairs as in-context demonstrations. This version of the problem is referred to as **source-to-source (s2s)**. The **source-to-target (s2t)** variant uses $sim(y_{i_j}, x)$ instead of $sim(x_{i_j}, x)$. It is possible to use a mix (**mix**) of both with $\alpha~sim(x_{i_j}, x) + (1 - \alpha)~sim(y_{i_j}, x), \alpha \in ]0, 1$. We consider sentence embeddings (SE) based similarity metrics (cosine similarity between sentence representations): [SONAR, Cohere (Embed V3), LaBSE, Laser 2 and E5. Additionally, we include BLOOM 7B1 based embeddings obtained by considering the token-wise average hidden state or the last token's hidden state at the first, the middle or the last layer. In our experiments, we worked with FLORES-200 using the `devtest` for evaluation and the `dev` as the selection pool. The `dev` vectors for each embedding method mentionned above are pre-computed for English (eng), French (fra), German (deu), Swahili (swh) and Wolof (wol) and can be …