IamNaodW/Amharic-Stemmer

# Amharic Stemmer — First-Pass Scaffold An iterative, longest-match, rule-based affix-stripping stemmer for Amharic, following the design pattern used in published Amharic/Tigrinya/Afaan Oromo stemmers (normalize → strip prefixes → strip suffixes → optional dictionary verification). ## Files - `normalize.py` — folds historically-distinct but phonetically-merged Ge'ez character series (ሐ/ኀ→ሀ, ሠ→ሰ, ዐ→አ, ፀ→ጸ) so identical words match consistently before affix rules are applied. - `affixes.py` — categorized prefix/suffix rule tables (prepositional clitics, verb subject/aspect prefixes, negative circumfix, object and possessive pronoun suffixes, plural/case markers). **Starting point only.** - `stemmer.py` — the stripping algorithm itself, plus an `is_valid_stem` dictionary-verification hook (no-op until you supply a lexicon). - `test_stemmer.py` — a handful of worked examples with expected rough segmentations, for sanity-checking, not a gold-standard evaluation set. ## Known limitation, demonstrated on purpose Try `python3 stem_cli.py "ትምህርት ቤት"` (no lexicon): the stemmer incorrectly strips `ት` off `ትምህርት` ("education/school"), a noun that happens to start with a syllable that's also a verb subject prefix, corrupting the stem to `ምህር`. This is a genuine over-stemming failure caused by having no part-of-speech awareness and no dictionary to check against. Now run `python3 stem_cli.py --lexicon example_lexicon.txt "ትምህርት ቤት"` — with even a tiny reference lexicon, the algorithm checks whether the *current* word is already a confirmed real stem before attempting to strip it further, and correctly leaves `ትምህርት` alone. This dictionary-backed back-off is the single highest-leverage addition you can make; scale `example_lexicon.txt` up with real stems from your corpus and accuracy will improve substantially. No rule-based system covers every word — expect it to make more mistakes on words it's never effectively seen a matching lexicon entry for. "Works for any word" her …