Redskull23/low-resource-safety-eval

Make AI safety multilingual. Policies → prompts → results → SAR/ASR/FRR, with CC0 mini-corpora and adversarial tests for low-resource languages. # low-resource-safety-eval A compact, public demo of **language inclusion + safety evaluation** for **low-resource** and **multilingual** LLM use cases. This repo shows an end-to-end **policy → prompts → model run → metrics → report** pipeline you can fork, extend, and discuss with stakeholders—**without** claiming deep linguistics coverage. It’s designed for product thinking, safety program design, and lightweight research prototyping. --- ## Why this is useful - **Language inclusion**: Goes beyond English to cover low-resource languages, highlighting where safeguards silently fail due to code-mixing, idioms, or orthography. - **Product thinking demo**: Turns abstract “we need safety” into a reproducible pipeline and single-file metrics you can put in PRDs, RFCs, or readouts. - **Transparent data**: Mini corpora are **synthetic** and **CC0**; provenance and consent notes are explicit by design. - **Portable**: Simple CSV in/out; works with any model/API; optional GitHub Actions runs metrics on every PR. - **Extensible**: Drop in new languages, categories, adversarial attacks, or policies as YAML and JSONL—no refactor. --- ## What’s inside *Languages included:* `ht`, `yo`, `en`, `de`, `fr`, `es`, `ja`,`gn`,`wo` ```text . ├── .github/ # GitHub Actions CI, issue/PR templates ├── .vscode/ # VS Code settings & extension recs ├── adversarial/ # Jailbreak / jiggle prompts (JSONL) ├── data/ # CC0, synthetic mini-corpora + notes │ ├── ht/ (Haitian Creole) ├── yo/ (Yorùbá) │ ├── en/ (English) ├── de/ (German) │ ├── fr/ (French) ├── es/ (Spanish) │ └── ja/ (Japanese) ├── docs/ # Annotation guidelines & context notes ├── notebooks/ # Metrics notebook (exploratory) ├── policies/ # YAML safety categories & markers ├── reports/ # Generated prompts & model results (CSV/JSON) ├── sc …