grmawit143/tigrigna-hybrid-stemmer-miku

Hybrid Tigrigna stemming system using Rule-based and BiLSTM model # 📌 Hybrid Tigrigna Stemming System (Miku) ## 📖 Overview This project presents a hybrid stemming system for the Tigrigna language, integrating a context-free rule-based stemmer with a context-aware BiLSTM neural network enhanced with attention mechanism. The system addresses key morphological challenges in Tigrigna, including: - Inflectional and derivational variations - Reduplication (single, double, and partial) - Irregular word forms - Context-dependent ambiguity The hybrid approach combines linguistic rules with deep learning to improve stemming accuracy in a low-resource language setting. --- ## 🎯 Objectives - Improve stemming accuracy for Tigrigna text - Combine rule-based linguistic knowledge with deep learning - Handle complex morphological structures - Build a reproducible NLP framework for low-resource languages --- ## ⚙️ Methodology ### 1. Rule-Based Stemmer - Removes prefixes and suffixes - Handles reduplication patterns - Processes regular word forms ### 2. BiLSTM Model - Character-level embeddings - Sentence-level context learning - Handles irregular and ambiguous words - Uses attention mechanism ### 3. Hybrid Approach Pipeline: 1. Rule-based stemming 2. BiLSTM refinement 3. Confidence-based decision --- ## 📊 Dataset - Total tokens: 32,151 Tigrigna words Categories: - Regular words - Irregular words - Ambiguous words - Stop words - Reduplicated forms (single, double, partial) Split: - 70% Training - 15% Validation - 15% Testing --- ## 📈 Experimental Results | Method | Accuracy (%) | Precision | Recall | F1-score | Balanced Accuracy | Macro F1 | Rank | | ---------- | ------------ | --------- | ------ | -------- | ----------------- | -------- | ------ | | **Hybrid** | **90.71** | **0.9390** | **0.9071** | **0.9167** | **0.8768** | **0.7732** | ★ BEST | | BiLSTM | 64.65 | 0.7196 | 0.6465 | 0.6607 | 0.6354 | 0.4541 | • | | Rule-Based | 51.32 | 0.6178 | 0.5132 | 0.5369 | 0.4194 | 0.2384 | • | --- ## 🔬 …