EN-AenaHabib/AenPi

A lightweight and efficient NLP framework designed for low-resource language processing, built with a strong focus on Green AI principles, including energy-efficient computation, reduced carbon footprint, and optimized modular design for real-world deployment. It provides reusable components for preprocessing, analysis, and modeling. # AenPi ### A Lightweight Green-AI NLP Library for Urdu *The first UNIFIED, pip-installable Urdu NLP library that runs entirely on CPU — no transformers, no GPU, no LLM API calls required.* Installation · Quick Start · Modules · Pipeline · Green AI · Benchmarks --- ## Why AenPi Urdu is spoken by over **230 million people**. Yet every modern NLP tool either ignores it entirely or forces you into one of two bad choices: - **Legacy tools** (UrduHack, iNLTK) — incomplete, undocumented, broken on real-world text - **Transformer models** (mBERT, XLM-R, UrduBERT) — 270M+ parameters, GPU required, 500ms+ latency per call, and enormous energy cost AenPi is built around a third option: **classical, interpretable, efficient methods** (CRFs, finite-state rules, TF-IDF, logistic regression) that run on any laptop, in any classroom, with zero API dependency , no cost . ``` Installed size --- ## Installation **From GitHub (recommended for Colab):** ```bash pip install git+github.com ``` **For local development:** ```bash git clone github.com cd AenPi pip install -e . ``` **Dependencies:** ```bash pip install scikit-learn datasets requests numpy psutil # Optional — for CRF-based POS and NER: pip install sklearn-crfsuite ``` --- ## Module Reference ### Preprocessor ```python from AenPi import urdu urdu.preprocess(text, remove_diac=True, remove_non_urdu_chars=True) # Full pipeline: removes diacritics, non-Urdu chars, punctuation, extra spaces urdu.tokenize(text) # Split text into word tokens → list urdu.remove_punctuation(text) # Strip Urdu and common punctuation ``` --- ### Normalizer Handles Unicode normalization, Alef/Ya/Kaf/Ha character variants, ligatures, and punctuation spacing. ```python norm = UrduNormalizer() print(norm.normalize("ﻣﯿﺮﺍ ﻧﺎﻡ احمد هے")) # Full normalization → str from AenPi.urdu import UrduNormalizer ``` --- ### Roman Urdu Normalizer Maps the 50+ spelling variants of Ro …