mikiyasegaye/Amharic-E-commerce-Data-Extractor

B5W4: Building an Amharic E-commerce Data Extractor --- # EthioMart Amharic E-commerce NER System A complete machine learning pipeline to extract structured information—like product names, prices, and locations—from Amharic e-commerce Telegram messages. This helps EthioMart assess vendors, track pricing, and identify business opportunities from unstructured chat data. --- ## 🚀 Project Goal To build a full pipeline that: - Collects Telegram posts from e-commerce channels - Preprocesses Amharic text - Labels key entities manually - Trains and compares NER models - Explains predictions (Model Interpretability) - Scores vendors for micro-lending (FinTech use case) --- ## 🔧 Tools and Technologies - Python 3.11+ - Telethon for Telegram scraping - Transformers (HuggingFace) - pandas, numpy - Jupyter Notebooks - SHAP, LIME for interpretability - scikit-learn for scoring - tqdm, evaluate, datasets - PyTorch with MPS for Apple Silicon --- ## 🗂️ Directory Structure ```bash . ├── config/ # YAML config files ├── data/ │ ├── raw/ # Raw Telegram messages │ └── processed/ # Cleaned and labeled data (CoNLL format) ├── notebooks/ # All development notebooks │ ├── fine_tune_ner_model.ipynb # Training NER model │ ├── compare_models.ipynb # Model benchmarking │ ├── interpret_ner_model.ipynb # SHAP & LIME explanation │ ├── vendor_scorecard.ipynb # Vendor analytics │ └── exploration.ipynb # Text cleaning demo ├── preprocessing/ │ ├── amharic_text_cleaner.py │ └── preprocess_pipeline.py ├── src/ │ ├── data_ingestion/ # Scraper logic │ ├── labeling/ # CoNLL formatting │ ├── modeling/ # Training, evaluation helpers │ └── vendor_analysis/ # Lending score logic ├── utils/ # Reusable I/O, logging helpers ├── tests/ # Unit tests ├── requirements.txt └── README.md ``` --- ## ✅ How to Run the …