bikoendalew/amharic-normalizer

Normalize Amharic homophones for accurate text search and NLP pipelines. # amharic-normalizer > Normalize Amharic homophones so phonetically identical words match regardless of how they were typed. --- ## The Problem The Ethiopic script contains **multiple distinct characters for identical sounds**. The "Ha" sound alone can be written as `ሀ`, `ሐ`, or `ኀ` — they look different, but a native speaker would never notice the difference. This causes a silent, critical bug in any system that stores or searches Amharic text: ``` User searches for: ሀብታሙ ← typed with ሀ Database stores: ሐብታሙ ← stored with ሐ Result: 0 results found ✗ ``` **amharic-normalizer** fixes this by mapping every group of phonetically equivalent characters to one canonical form before comparison. --- ## Installation ```bash pip install amharic-normalizer ``` --- ## Quick Start ```python from amharic_normalizer import normalize user_query = "ሀብታሙ" # typed with ሀ database_entry = "ሐብታሙ" # stored with ሐ # Without normalization user_query == database_entry # False ✗ # With normalization normalize(user_query) == normalize(database_entry) # True ✓ ``` --- ## Homophone Groups | Sound | Redundant family | Canonical | Example | |-------|-----------------|-----------|---------| | **Ha** | `ሐ ሑ ሒ ሓ ሔ ሕ ሖ` and `ኀ ኁ ኂ ኃ ኄ ኅ ኆ` | `ሀ` family | `ሐብታሙ` → `ሀብታሙ` | | **Se** | `ሠ ሡ ሢ ሣ ሤ ሥ ሦ` | `ሰ` family | `ሥራ` → `ስራ` | | **Tse** | `ፀ ፁ ፂ ፃ ፄ ፅ ፆ` | `ጸ` family | `ፀሐይ` → `ጸሀይ` | ### Optional: Ain → Alef In classical Ge'ez and Tigrinya, `ዐ` (pharyngeal ain) and `አ` (glottal stop) are distinct phonemes. In modern Amharic they sound identical. This merge is **opt-in**: ```python normalize("ዐማራ", normalize_ain=True) # → "አማራ" ``` --- ## API ### `normalize(text, *, normalize_ain=False) → str` Replaces redundant Ethiopic characters with their canonical equivalents. All non-Ethiopic characters (Latin, digits, punctuation) pass through unchanged. ```python from amharic_normalizer import normalize normalize("ፀሐይ") …