wondifraw/Amharic-E-commerce-Data-Extractor

# Amharic NER for E-commerce Analytics An enterprise-grade Named Entity Recognition (NER) system for Amharic e-commerce data analysis, featuring automated Telegram channel scraping, multi-model training, and comprehensive vendor analytics for micro-lending risk assessment. ## 🚀 Key Features ### Core Capabilities - **🔄 Data Pipeline**: Automated Telegram channel scraping with rate limiting - **📝 Text Processing**: Amharic-specific preprocessing and normalization - **🏷️ Smart Labeling**: CoNLL format annotation with semi-automated labeling - **🤖 Multi-Model Training**: Fine-tuning of XLM-RoBERTa, DistilBERT, and mBERT - **📊 Model Evaluation**: Comprehensive performance comparison and selection - **🔍 Interpretability**: SHAP and LIME model explanations - **💼 Vendor Analytics**: Risk assessment and micro-lending scorecard generation ### Business Intelligence - Real-time vendor performance tracking - Market trend analysis and insights - Automated risk scoring (0-100 scale) - Interactive analytics dashboard - Export capabilities (PDF, Excel, JSON) ## 🛠️ Installation & Setup ### Prerequisites - Python 3.9+ - Telegram API credentials - 8GB+ RAM (for model training) ### Quick Start ```bash # Clone repository git clone github.com cd Amharic-E-commerce-Data-Extractor # Setup environment pip install -r requirements.txt cp .env.example .env # Configure credentials (edit .env file) # TELEGRAM_API_ID=your_api_id # TELEGRAM_API_HASH=your_api_hash # Run full pipeline python main_pipeline.py ``` ### Quick Run for All Tasks ```bash # 1. Data Collection python -m src.data_ingestion.telegram_scraper # 2. Data Preprocessing python -m src.preprocessing.text_cleaner # 3. Data Labeling python -m src.labeling.conll_formatter # 4. Model Training python -m src.training.train_models # 5. Model Evaluation python -m src.evaluation.compare_models # 6. Model Interpretability python -m src.interpretability.explain_models # 7 …