NuryeNigusMekonen/Amharic-E-commerce-Data-Extractor

Transform messy Telegram posts into a smart FinTech engine that reveals which vendors are the best candidates for a loan. ## Amharic Named Entity Recognition (NER) A deep learning pipeline for Named Entity Recognition on **Amharic e-commerce data**, using multilingual Transformer models with **CRF decoding**, **weighted loss for class imbalance**, and **interpretability tools** like **SHAP** and **LIME**. --- ## Project Structure ``` Amharic-E-commerce-Data-Extractor/ │ ├── data/ # Raw and labeled data (e.g., .conll format) ├── models/ # Saved model checkpoints ├── notebooks/ # Interactive notebooks for training and experiments ├── src/ # Main source code │ ├── config.py # Config: paths, labels, model list, weights │ ├── model.py # CRF-enhanced Transformer model │ ├── prepare_dataset.py # Dataset loading and preprocessing │ ├── train.py # Model training script │ ├── predict.py # Inference and post-processing │ ├── evaluate_models.py # Model comparison and selection │ └── interpret.py # SHAP & LIME interpretability ``` --- ## Features * **Multilingual Transformers**: `XLM-R`, `mBERT`, `AfroXLMR`, `BERT-Tiny-Amharic` * **CRF Layer** for better sequence modeling * **Weighted Loss** to handle label imbalance * **BIO Tag Postprocessing** to fix tagging errors * **Model Comparison**: Accuracy, F1, and robustness * **Explainability**: LIME & SHAP for token-level insights --- ## Installation ```bash git clone github.com cd Amharic-E-commerce-Data-Extractor # Create and activate virtual environment python -m venv AE-venv-py310 source AE-venv-py310/bin/activate # Install dependencies pip install -r requirements.txt ``` --- ## Labels The NER system supports 9 classes (BIO format): * `B-PRODUCT`, `I-PRODUCT` * `B-PRICE`, `I-PRICE` * `B-LOC`, `I-LOC` * `B-PHONE`, `I-PHONE` * `O` – Outside entity --- ## Training Train and save models with CRF layer: …