dai-michael/English-Amharic-Translation-Project

English–Amharic machine translation research project with Prof. Mark Hopkins and current and former student researchers # English-Amharic Translation Project Translation project conducted with guidance from Prof. Mark Hopkins at Williams College. The repository combines contributions from Aunonto Rahman on contrastive preference optimization (CPO), Adhip Rijal on prefix tuning and OFT (PEFT and manual) implementations, and Michael Dai on decoder-only model training and LoRA. Built on work from previous student researchers working with Prof. Mark Hopkins, including Denis Sotnichenko. ## Setup Copy `local_env.sh.example` to `local_env.sh` and set paths for your environment. Install dependencies from your existing environment (transformers, peft, torch, evaluate, etc.). Update corpus paths in your config JSON before running. ## Training entry points | Experiment type | Script | Config flags | |-----------------|--------|--------------| | Encoder-decoder SFT / LoRA / full FT | `finetune.py` | Standard `bitexts` with `src`/`tgt` | | CPO (contrastive preference optimization) | `finetune_cpo.py` | `cpo: true`, bitext with `src`/`good`/`bad` | | Manual prefix / OFT / stacked adapters | `finetune.py` | `use_prefix_tuning`, `use_oft`, `use_lora` | | Decoder-only SFT | `finetune_decoder.py` | `model_type: "decoder"`, `train_type: "sft"` | | Decoder continual pretraining (CPT) | `finetune_decoder.py` | `model_type: "decoder"`, `train_type: "continual_pretrain"` | | Decoder trainable token embeddings | `finetune_decoder.py` | `model_type: "decoder"`, `train_type: "trainable_tokens"` | Example configs are in `examples/example_configs/`. Example SLURM job scripts are in `examples/slurm/`. ## Encoder-decoder experiments On the cluster you typically cannot run training by invoking `python` directly in an interactive terminal — jobs need a GPU and should be submitted through SLURM. Write a shell script (or use one from `examples/slurm/`) with `#SBATCH` headers, `source local_env.sh`, and the `python ... --config ...` command, then submit with `sbatch`. The `python` commands below show what g …