Teshager21/amharic-telegram-ecommerce-ner

# Amharic Telegram E-commerce NER 🚀 **Amharic Telegram E-commerce NER** is a complete end-to-end NLP pipeline for extracting structured product data from Amharic-language Telegram-based e-commerce channels. It powers the EthioMart platform by fine-tuning transformer-based models to recognize key business entities like **Product Names**, **Prices**, and **Locations** in unstructured messages. --- ## 📌 Project Goals - 📥 Ingest real-time text and image data from multiple Amharic Telegram e-commerce channels. - 🧼 Preprocess and structure raw Amharic text data for downstream NLP tasks. - 🧠 Fine-tune multilingual transformer models (e.g., XLM-R, mBERT) for **Named Entity Recognition (NER)** in Amharic. - 🧪 Compare model performance using F1-score, precision, recall. - 🔍 Interpret predictions with SHAP and LIME to ensure transparency. - 📊 Score vendors using engagement metrics + extracted business info for micro-lending insights. --- ## 🛠️ Tech Stack | Layer | Tools / Libraries | |-------------|------------------------------------------------| | Data Ingestion | `telethon`, `tdlib` for Telegram scraping | | NLP & NER | `transformers`, `datasets`, `seqeval`, `XLM-R` | | Tracking | `MLflow`, `DVC` | | Annotation | `CoNLL`, manual labeling, `doccano` (optional) | | Explainability | `SHAP`, `LIME` | | Deployment | Google Colab, local, or cloud | --- ## 📦 Features - 🐦 **Real-time Telegram data extraction** - 🔠 **Amharic text normalization & tokenization** - 🏷️ **Custom NER dataset in CoNLL format** - 🤖 **Fine-tuned transformer models for NER** - 📈 **Evaluation & model comparison dashboards** - 💡 **Interpretable outputs using SHAP/LIME** - 💰 **Vendor scorecard engine for micro-lending insights** --- ## 🗂 Project Structure amharic-telegram-ecommerce-ner/ ├── data/ │ ├── raw/ │ ├── processed/ ├── src/ │ ├── ingestion/ # Telegram scraping logi …