Jeff-2000/kreyolbench

Open-source benchmark suite for Haitian Creole (Kreyòl Ayisyen) NLP, featuring standardized datasets, evaluation tasks, baselines, and reproducible benchmarks for low-resource language research. # KreyolBench KreyolBench is a Kreyol-first benchmark scaffold for Haitian Creole NLP. It is designed for reproducible research, Hugging Face compatibility, and public release with explicit source, license, and provenance tracking. The benchmark prioritizes real-world Haitian Creole use cases: education, public health, civic information, disaster response, administration, translation, search, text normalization, and digital inclusion. ## Quick Start ```bash python -m venv .venv source .venv/bin/activate pip install -e ".[dev]" kreyolbench validate-dataset --task ner --path data/sample/ner.jsonl kreyolbench evaluate --task classification --predictions data/sample/classification_topic.jsonl --references data/sample/classification_topic.jsonl pytest ``` ## Tasks Version `v0.1` targets topic/domain classification, NER, extractive QA, retrieval, text normalization, code-switch sentence/token labels, and translation metadata evaluation. Later releases add sentiment, summarization, orthographic robustness, hidden tests, and larger domain transfer settings. ## Data Policy This repository does not assume external datasets are redistributable. Source adapters and registry files record public HTTPS locations, access methods, licenses, and review status. Raw external data should be stored under `data/raw/` locally and should not be committed unless redistribution is explicitly allowed. ## Collaboration and Security KreyolBench uses staged collaboration and least-privilege access. New collaborators should start with a small non-sensitive task before receiving broader access. - Project mission and ethical boundaries: `PROJECT_CHARTER.md` - Governance and roles: `GOVERNANCE.md` - Contribution workflow: `CONTRIBUTING.md` - Conduct expectations: `CODE_OF_CONDUCT.md` - Security and incident reporting: `SECURITY.md` - Authorship and credit: `AUTHORSHIP.md` - Restricted data rules: `DATA_ACCESS_POLICY.md` - Full operating guide: `docs/collaboration_security_guide.md` - Inta …