wxjiao/WMT2022-Large-Scale-African

Introduction to "Tencent’s Multilingual Machine Translation System for WMT22 Large-Scale African Languages". # WMT2022-Large-Scale-African This project introduces Tencent’s Multilingual Machine Translation System for the WMT2022 Large-Scale African Translation shared task. We (team Borderline) won the 1st place :1st_place_medal: in the constrained track (i.e., external data and pretrained models are not allowed). Competition Results on the Blind Test Sets (By 2022/09/29) ## Test our Pretrained Model ### Config the Local Environment for Evaluation Let us assume we are provided with a clean machine, then we have the following steps: - Download Miniconda3 py37_4.10.3, and see Instruction to install. ``` # make sure your networks are well configured. wget repo.anaconda.com sh Miniconda3-py37_4.10.3-Linux-x86_64.sh # three prompts during installing # 1. Do you accept the license terms? [yes|no] # yes # 2. Miniconda3 will now be installed into this location: /home/username/miniconda3 # ENTER # 3. Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no] # yes source ~/.bashrc ``` - Install Pytorch 1.7.1 + CUDA 11.0. ``` # CUDA 11.0 pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f download.pytorch.org ``` - Install `fairseq==0.10.2` and dependencies with `sub_requirements.txt`. ``` pip install -r sub_requirements.txt # packages in sub_requirements.txt tensorboardX cffi cython dataclasses hydra-core numpy regex sacrebleu tqdm sacremoses nltk>=3.2 matplotlib absl-py sentencepiece setuptools>=18.0 fairseq==0.10.2 ``` ### Download our Pretrained Model - Due to the storage limit, we only release the contrastive submission, which include our code, sentencepiece model, and pretrained model. ``` # decompress and obtain `submission_contrast` unzip submission_contrast.zip ``` - Use the `inference_afr.sh` for inference. ``` # update the `inference_afr.sh` file with the one provided in this repo cd submission_contrast pip in …