AugustinSamier/SCaMD

SCaMD (Similarity Chain-across-Multilingual Dictionaries) is a LLM-based framework exploring the combination of compositional-translation and the usage of self-generated dictionaries in the field of low-resource languages machine translation. SCaMD (Similarity Chain-across-Multilingual Dictionaries) is a LLM-based framework exploring the combination of compositional-translation and the usage of self-generated dictionaries in the field of low-resource languages machine translation. We use CompTra code : github.com Related with their study arxiv.org SCaMD pipeline: The following command applies $SCaMD-Core$ on flores200 dataset from English to Amharic with LLaMA 3.1 8B Instruct and save the results in a file: "SCaMD-Core/flores-English-Amharic-generations.jsonl". ```python python -m scamd_sampler\ --src "English"\ # Source language (e.g., English) --tgt "Amharic"\ # Target language (e.g., Amharic) --k 5\ # Number of nearest examples to retrieve --CodLangs ["French","German","Portuguese"]\ # Languages to build multilingual dictionaries --ds "flores"\ # Dataset name or path --checkpoint 0\ # Start index in the dataset --path_output "SCaMD-Core"\ # Output directory for generations --nb 100\ # Number of sentences to generate (None = all) --comptra True\ # Whether to run compositional translation --scamd True\ # Whether to run dictionary generation (SCaMD) --prompts_path None\ # Reuse saved prompts (None = regenerate) -> Example: "flores-Compositional-translation-Amharic" --outPrompts False\ # Output the prompts to file (without translation) --dictionaries_path None\ # Reuse saved dictionaries (None = regenerate) -> Example: "flores-Dictionaries-(English-French-German-Portuguese)" --outCod False\ # Output the dictionaries to file (without generation) --filterSize 40\ # Max word window repetition filter --filterChrSize 5\ # …