prakharmodi26/booster_jailbreak

Multilingual jailbreak attacks on Booster-defended LLMs (Mistral-7B) — French, Yoruba, Yoruba code-switch attack variants with ASR evaluation # Multilingual Jailbreak Attacks on Booster-Defended LLMs Tests robustness of **Booster** safety alignment against fine-tuning attacks using multilingual unsafe data — French (high-resource), Yoruba (low-resource), and Yoruba code-switched English. Base model: Mistral-7B-v0.1. ## Key Results | Variant | ASR | vs. M1 baseline | |---|---|---| | Raw (no defense) | 67.4% | — | | M1 (Booster defended) | 0.9% | baseline | | M2 French attack | **21.3%** | +23.7× | | M2 Yoruba code-switch | 3.6% | +4× | | M2 Yoruba pure | 1.6% | +1.8× | Booster cuts ASR 67.4% → 0.9% (−98.7%). French attack most effective — high-resource language, strong Mistral-7B representation. Full breakdown: `results/RESULTS_SUMMARY.md` --- ## Reproducing Results ### Prerequisites - Mistral-7B-v0.1 downloaded to `/workspace/raw_model/` (`mistralai/Mistral-7B-v0.1` on HuggingFace) - Booster repo cloned to `/workspace/Booster/` (`git clone github.com`) - CUDA GPU ≥40GB VRAM (tested on A100) - Python deps: `pip install -r /workspace/Booster/booster_pip.txt` ### 1. Configure paths ```bash cp config.example.sh config.sh # Edit config.sh: set BOOSTER_DIR, OUTPUT_DIR, RAW_MODEL_PATH, HUGGINGFACE_TOKEN source config.sh ``` ### 2. Generate attack datasets (skip if using provided `data/` files) ```bash # Verify setup first python3 scripts/test_translation.py # NLLB smoke test python3 scripts/test_code_switch.py # spaCy span extraction test # Translate python3 scripts/translate_french.py # → beavertails_30k_train_french.jsonl python3 scripts/translate_yoruba.py # → beavertails_30k_train_yoruba.jsonl python3 scripts/code_switch_yoruba.py # → beavertails_30k_train_codeswitch_yoruba.jsonl # Convert to instruction format python3 scripts/convert_data.py --input --output data/beavertails_french_instruct.json python3 scripts/convert_data.py --input --output data/beavertails_yoruba_instruct.json python3 scripts/convert_data.py --input - …