Logo Lanfrica
  • Accueil
  • Atlas
  • Analyses
  • Documentation
  • Sign in

© 2026 Lanfrica. Tous droits réservés. Tous les droits d'auteur des ressources affichées sur le site Web Lanfrica appartiennent aux détenteurs de droits d'auteur d'origine, sauf indication contraire explicite.

Natashadonoh11/yoruba-diacritic-restoration

Domaine:

natural language processing

Type de record:

datasetmodel
Créateur:
Nat
Hôte:
Yoruba diacritic restoration- Adaption Labs submission # Yorùbá Diacritic Restoration Adapting a low-resource dataset and fine-tuning a model to restore tone marks and underdots in undiacriticized Yorùbá text — submitted to Adaption Labs' AutoScientist. **Author:** Areo Mafadesere Natasha,Africomplings Summer School 2026 --- ## Overview Yorùbá orthography relies on tone marks (acute/grave/unmarked) and underdots (ẹ, ọ, ṣ) to disambiguate meaning. These diacritics are frequently dropped in informal digital text, which motivates **diacritic restoration** — recovering the correctly marked form from stripped input. This project builds a diacritic-restoration dataset from the MENYO-20k corpus, adapts it using Adaption Labs, and fine-tunes a model via AutoScientist. ## Links - 🤗 Model: Natashadonoh/yoruba-diacritic-restoration - 🤗 Dataset: Natashadonoh/yoruba-diacritic-restoration-dataset - 📄 Paper: (in this repo) ## Pipeline 1. **Convert** — `convert_menyo_tsv.py` parses MENYO-20k's `train.tsv` into a clean `english`/`yoruba` CSV. 2. **Annotate** — `annotate_proverbs.py` adds linguistic annotation columns: - `tone_pattern` — per-vowel High/Mid/Low tone sequence - `harmony_class` / `harmony_breakdown` — word-level ±ATR vowel harmony - `focus_tag` — heuristic flag for candidate *ni*-focus constructions 3. **Strip** — `strip_diacritics.py` removes tone marks and underdots via NFD decomposition to produce the undiacriticized `yoruba_stripped` column (the training input). 4. **Adapt** — the raw, annotated dataset is expanded/adapted via Adaption Labs (Instruction Tuning recipe). 5. **Fine-tune** — the adapted dataset is used to fine-tune a model via AutoScientist, with `yoruba_stripped` as prompt and `yoruba` as completion. ## Dataset - **Raw**: 8,365 rows, sourced from MENYO-20k (multi-domain: news, TED talks, book excerpts, ICT, proverbs; JW-sourced content excluded) - **Adapted**: 5,000 rows, generated via Adaption Labs' adaptation pipeline - **License**: CC BY-NC 4.0 (inherited from MENYO-20k) Full dataset descr …

Visit

github.com

Tasks

diacritic restorationtext normalization

Languages

Yoruba

Similaires

majtom91/yoruba-diacritic-restorationayomidedavid/yoruba-diacritic-restoration-modelDiacritic Restoration in Yoruba LanguageDiacritic Restoration for Yoruba Text with under dot and Diacritic Mark Based on LSTMCrinmatic/Diacritic-RestorationImproving Yorùbá Diacritic Restoration

majtom91/yoruba-diacritic-restoration

Automatic Yorùbá diacritic restoration application # Yoruba-diacritic-restoration Automatic Yorùbá

ayomidedavid/yoruba-diacritic-restoration-model

Yoruba Diacritic Restoration — Hybrid BiLSTM + Transformer This project provides a scaffold to trai

Diacritic Restoration in Yoruba Language

A project on diacritic restoration of languages

Diacritic Restoration for Yoruba Text with under dot and Diacritic Mark Based on LSTM

Abstract Yoruba is a tonal language spoken primarily in Nigeria, some West African countries,

Crinmatic/Diacritic-Restoration

Using AI to restore Diacritics on Yoruba language (which is a low resource language) ## **Diacritic

Improving Yorùbá Diacritic Restoration

Yorùbá is a widely spoken West African language with a writing system rich in orthographic and tonal diacritics. They provide morphological information, are crucial for lexical disambiguation, pronunciation and are vital for any computational Speech or Natural Lang