Yabsera-Haile/Human-vs-Machine-Translation-Detection-Codebase

This repository is used for reproducibility of a study of human vs machine translation detection across domains, models, and low-resource languages, combining fine-tuned and LLM methods with quality-aware evaluation to assess when machine translations become indistinguishable from human ones. # Human vs Machine Translation Detection (Cross-Model/Cross-Domain/Low-Resource) This repo supports reproducible experiments to detect machine-generated translation vs human translation. It includes data preparation, model fine-tuning, inference generation, and multi-metric quality analysis across: - languages: Afrikaans (af), English (en), Spanish (es), Swahili (sw) - datasets: FLORES, EU DGT, OPENWHO, SMOL - models: LLM and dedicated translation-detection models - data split: train/val/test and cross-domain evaluation sets --- ## 📁 Root files - README.md (original brief project description) - Data (experiment datasets and evaluation results) - `Dataset Generation/` (script notebooks for building data) - Fine-tuning (notebooks for training translation/model detector) - Inference (runtime generation and detection evaluation notebooks) - `Quality Evaluation/` (analysis notebooks with metrics and correlations) --- ## 📂 Data ### Structure - Data - `eval/` (evaluation-ready test outputs, predictions, model comparators) - `train/` (train/val csvs split by language) ### `eval` subsets - `eu_dgt_csvs/` - `en/`, `es/` - files: - `eu_dgt_output_with_models.csv` etc: model outputs merged with ref/human - `eu_dgt_test_{model}.txt`: baseline output of each MT model - `eu_dgt_model_mdeberta_sent_col_human translation.csv`: reference human translation - `flores/` - `af/`, `sw/` - same pattern: `flores_output_with_models*.csv`, `flores_test_{model}.txt`, `flores_test.csv`. - `openwho_csvs/` and `smol_csvs/` (similar) ### `train` - `train/af/train.csv`, `train/af/val.csv` - `train/en/train.csv`, `train/en/val.csv` - `train/es/train.csv`, `train/es/val.csv` - `train/sw/train.csv`, `train/sw/val.csv` Purpose: - training detector/classifier on bilingual pairs or feature engineered data - validation for tuning accuracy - test for final evaluation in eval --- ## 🧩 Dataset Generation - `Dataset Generation/Data Preprocessing/` notebooks: - `build_eu_dgt_pal.ipynb` - `build_sw …