Unkadlabs/unkad-guard-collapse

Do safety classifiers stop discriminating in Somali? Three guard models catch 100% of harmful English prompts and as few as 6% of the same prompts in Somali. Pre-registered, informedness-based. # Do safety classifiers stop discriminating in Somali? **Three open guard models, two languages, 300 prompts. Same harmful requests in English and Somali, measured with a metric that can tell a working filter from a dead one.** By Unkad Labs, a non-profit AI research laboratory in Mogadishu. Runs locally on quantised models through Ollama, no GPU and no API keys. > **Result: they collapse toward letting harm through.** Llama Guard 3 catches > 100% of harmful English prompts and 16% of the same prompts in Somali. > ShieldGemma, 67% to 6%. All three stay near-perfect on benign Somali, so this > is a specific failure to recognise harm, not a general failure to read the > language. Read the writeup: The guard that waves Somali > through. ## The question Guard models (Llama Guard, ShieldGemma, Granite Guardian) are the safety filters in front of deployed language models. The literature on how they fail in low-resource languages disagrees: some work reports over-blocking, some reports leniency, and none of it covers Somali. We measured which way three of them fail, using a metric the field does not report. The instrument matters. On a realistic, mostly-benign traffic mix a guard that has stopped catching harm still posts high accuracy, because accuracy is dominated by the easy benign majority. So the primary metric here is **informedness** (Youden's J = TPR + TNR - 1), which is 0 when a verdict carries no information about the true label, regardless of class balance. ## Result Harmful catch rate (TPR), English vs Somali, n = 100 harmful + 50 benign per language: | guard | catch (EN) | catch (SO) | benign pass (SO) | informedness (EN → SO) | |---|---|---|---|---| | Llama Guard 3 | 1.00 | 0.16 | 0.96 | 1.00 → 0.12 | | ShieldGemma | 0.67 | 0.06 | 1.00 | 0.67 → 0.06 | | Granite Guardian | 1.00 | 0.63 | 0.98 | 1.00 → 0.61 | Pre-registered hypotheses: H1 (Somali informedness lower) **confirmed** on all three. H3 (English informedness above zero, the rig control) **pas …