baharia13/swahiliclean-R

swahiliclean is an R package that cleans and standardizes Swahili (Kiswahili) text for NLP and corpus work. It uses curated internal dictionaries only (no internet APIs, no AI spell-checkers). # swahiliclean: Swahili Text Preprocessing for R **Version 0.2.1** — An R package for comprehensive preprocessing of Swahili (Kiswahili) text data, designed for NLP tasks. Provides normalization, stopword removal, slang normalization, typo correction, tokenization, email/URL cleaning, character filtering, text statistics, a configurable full pipeline, and **batch corpus file processing**. ## Authors - **Bernard Masua** — bhrmasua@gmail.com - **Noel Masasi** — noeliasmasasi@gmail.com ## Installation Install from GitHub using the `remotes` package: ```r remotes::install_github("baharia13/swahiliclean-R") library(swahiliclean) ``` ### Web app (no R coding required) Install the UI dependencies once, then launch the browser interface: ```r install.packages(c("shiny", "bslib", "DT")) library(swahiliclean) run_swahiliclean_app() ``` Or with Docker: ```bash docker compose --profile shiny up --build ``` Then open localhost. Paste or upload text, set preprocessing options, run the pipeline or individual tools, compare before/after results, and download cleaned text or reports. ## Features - **Text Normalization**: Lowercase, trim, collapse whitespace (optional preserve newlines/tabs) - **Stopword Removal**: ~218 common Swahili stopwords (custom lists supported) - **Slang Normalization**: ~190 slang → standard mappings - **Typo Correction**: ~422 deterministic typo fixes - **Tokenization**: Word tokenization with optional custom regex pattern - **Email & URL Removal**: Detect and remove emails and URLs - **Character Filtering**: Non-alphanumeric, non-alphabetic, short words, emoji, non-ASCII - **Text Statistics**: Count words, stopwords, slang, typos with percentages - **Full Pipeline**: `preprocess_sw()` with configurable steps and custom dictionaries - **Corpus batch processing**: Clean whole files or folders (`preprocess_sw_file()`, `preprocess_sw_corpus()`) - **Dual API names**: `normalize_text()` / `normalize_sw_text()`, etc. ## Quick Start ``` …