2zalab/fulfulde-stopwords

A linguistically curated stopword list for Fulfulde (Adamawa variant), designed for NLP tasks in low-resource African languages. # Fulfulde Stopwords A curated list of **stopwords for the Fulfulde language** (Adamawa variant), designed for Natural Language Processing (NLP) applications. Fulfulde is a low-resource African language spoken by millions across West and Central Africa. This library provides the first comprehensive, linguistically grounded stopword list for NLP tasks such as: - Text classification - Information retrieval - Clustering and topic modeling - Text preprocessing and cleaning - Machine translation - Sentiment analysis ## Features - **Linguistically grounded**: Compiled by linguists and NLP researchers - **Comprehensive**: 180+ stopwords covering all major grammatical categories - **Easy to use**: Simple Python API with no external dependencies - **Well documented**: Extensive documentation and examples - **Tested**: Comprehensive test suite - **Open source**: MIT licensed ## Installation ### From PyPI (recommended) ```bash pip install fulfulde-stopwords ``` ### From source ```bash git clone github.com cd fulfulde-stopwords pip install -e . ``` ## Quick Start ### Basic Usage ```python from fulfulde_stopwords import get_stopwords, remove_stopwords, is_stopword # Get all stopwords stopwords = get_stopwords() print(f"Total stopwords: {len(stopwords)}") # Check if a word is a stopword print(is_stopword('mi')) # True print(is_stopword('wuro')) # False # Remove stopwords from text tokens = ['mi', 'heɓi', 'wuro', 'e', 'nder', 'Kameruun'] filtered = remove_stopwords(tokens) print(filtered) # ['heɓi', 'wuro', 'Kameruun'] ``` ### Advanced Usage ```python from fulfulde_stopwords import filter_text, get_stats # Filter an entire text text = "mi heɓi wuro e nder Kameruun" filtered_text = filter_text(text) print(filtered_text) # "heɓi wuro Kameruun" # Get statistics tokens = ['mi', 'heɓi', 'wuro', 'e', 'nder'] stats = get_stats(tokens) print(stats) # { # 'total_tokens': 5, # 'stopword_count': 3, # 'content_word_ …