maimouna-mbacke/wolof-nlp

A linguistically informed NLP toolkit for Wolof, designed to handle agglutinative morphology, complex TAM systems, and code-switching. # Wolof NLP Toolkit Wolof is a Niger–Congo language spoken by over 18 million people across Senegal, Gambia, and Mauritania, yet it remains largely unsupported by mainstream NLP libraries such as spaCy and NLTK. This toolkit addresses that gap by providing Wolof-aware processing that accounts for the language’s complex tense–aspect–mood (TAM) system, agglutinative morphology, and pervasive code-switching with French and Arabic. ## Features - **Tokenization** with two modes: word-level and morpheme-level - **Language detection** for Wolof, French, and Arabic loanwords - **Orthography normalization** (informal → CLAD standard) - **Morphological analysis** with derivational suffix recognition - **POS tagging** with Wolof-specific tagset - **Named Entity Recognition** with Senegalese gazetteers - **Sentiment analysis** with negation and intensifier handling - **Interlinear glossing** following Leipzig conventions ## Installation ```bash pip install wolof-nlp ``` Or install from source: ```bash git clone github.com cd wolof-nlp pip install -e . ``` ## Quick Start ```python from wolof_nlp import WolofTokenizer, tokenize, morphemes, normalize, analyze_morphology # Tokenization with language detection tokenizer = WolofTokenizer(normalize=True, detect_language=True) tokens = tokenizer.tokenize("Dafa trop neex") for tok in tokens: print(f"{tok.text}: {tok.language.name}") # Dafa: WOLOF # trop: FRENCH # neex: WOLOF # Two tokenization modes tokenize("Damay dem") # ['Damay', 'dem'] - word level morphemes("Damay dem") # ['da', 'ma', 'y', 'dem'] - morpheme level # Orthography normalization normalize("dieuradieuf") # → 'jërëjëf' # Morphological analysis analyze_morphology("bindkat") # → [bind:ROOT, kat:NOMINALIZATION] ``` ## Applications ```python from wolof_nlp.applications import tag, extract_entities, analyze_sentiment, gloss # POS Tagging tag("Xale bi dafa lekk") # [('Xale', 'NOUN'), ('bi', 'DET'), ('dafa', 'TAM'), …