michael-099/Amharic-E-commerce-Data-Extractor-10ac-w4

# Amharic E-commerce Data Extractor ## Project Overview This project implements a comprehensive Named Entity Recognition (NER) system for extracting key business entities (Product, Price, Location) from Amharic text in Ethiopian e-commerce Telegram channels. The system is designed to support EthioMart's vision of becoming a centralized hub for Telegram-based e-commerce activities in Ethiopia. ## Business Objectives 1. **Data Ingestion**: Automated collection from multiple Ethiopian e-commerce Telegram channels 2. **Entity Extraction**: High-accuracy NER for Product, Price, and Location entities in Amharic text 3. **Model Comparison**: Systematic evaluation of multiple transformer-based models 4. **Interpretability**: SHAP and LIME explanations for model predictions 5. **Vendor Analytics**: Micro-lending scorecard system for vendor assessment ## Project Structure ``` ├── Data/ # Data files and datasets │ ├── labeled_telegram_product_price_location.txt # Original labeled data │ ├── merged_labeled_data.txt # Combined training dataset │ └── channels_to_crawl.xlsx # Channel information ├── scripts/ # Core implementation scripts │ ├── scraper.py # Telegram data scraping │ ├── data_processor.py # Enhanced data preprocessing │ ├── conll_labeler.py # CoNLL format data labeling │ ├── ner_trainer.py # Model fine-tuning │ ├── model_evaluator.py # Model comparison framework │ ├── model_interpretability.py # SHAP/LIME explanations │ └── vendor_scorecard.py # Vendor analytics engine ├── models/ # Trained model artifacts ├── media/ # Downloaded images from channels ├── notebooks/ # Jupyter notebooks for analysis └── requirements.txt # Project dependencies ``` ## Key Features ### 1. Data Collection and Preprocessing - **Telegram Scraper**: Autom …