Logo Lanfrica

Script Is an Alignment Surface

Domaine:

natural language processing

Type de record:

paper
Créateur:
Saf
Éditeur:
Cen
Éditeur:
OSF
Hôte:avatar
Modern aligned LLMs refuse harmful Arabic prompts but comply with the same semantic content when rendered in Arabizi—the Latin-script transliteration of colloquial Arabic. Recent work has documented this asymmetry behaviorally across multiple language families (Al Ghanim et al., EMNLP 2024; Yoo et al., NAACL 2025; Banerjee et al., 2025; Jamil & Rafi, 2026) but has not explained its mechanism. This study investigates whether refusal alignment is mechanistically anchored to surface orthographic features rather than abstract semantic content. We extract refusal directions from Qwen2.5-7B-Instruct under matched-meaning, script-only-variant pairs (Arabic ↔ Arabizi), perform 2×2 causal cross-script interventions to test predicted asymmetry, and use Andy Arditi's seven-layer sparse-autoencoder release (layers 3, 7, 11, 15, 19, 23, 27) to produce the first depth-resolved analysis of orthographic-anchored alignment features. A second-language generalization test (Hindi Devanagari ↔ ITRANS Latin) establishes the effect's theoretical generality independent of language family. We pre-register five hypotheses with quantitative rejection criteria: - H1: Refusal rate gap between Arabic and Arabizi ≥ 25 percentage points (reject if < 10pp) - H2: Causal direction asymmetry (forward flip - reverse flip) ≥ 20pp (reject if < 10pp) - H3: Top-100 SAE features Jaccard similarity < 0.4 at layer 15 (reject if > 0.6) - H4: Cross-language generalization to Hindi ≥ 10pp asymmetry (reject if < 5pp) - H5: Wang's silhouette gap (0.24 vs 0.39 for Arabic vs English) narrows on Arabizi prompts The expected primary contribution is the first causal-mechanistic and depth-resolved explanation of cross-script safety failures, along with a reusable map of orthographic features in the residual stream. Full pre-registration document with detailed analysis plan, sample sizes, and rejection criteria is uploaded as osf_preregistration.md in this project's Files.