Yihenew21/Amharic-E-commerce-Data-Extractor

# 🛍️ Amharic E-commerce Data Extractor ## 📌 Project Overview The **Amharic E-commerce Data Extractor** is an end-to-end pipeline for transforming unstructured Telegram e-commerce posts (text and images) into structured, business-ready data using **Named Entity Recognition (NER)**. It extracts key entities — **Product**, **Price**, **Location**, **Brand**, **Size**, and **Contact** — to power EthioMart, a centralized Amharic e-commerce hub. The project addresses the challenge of decentralized commerce on Telegram by scraping, processing, labeling, and analyzing messages from various channels. It includes fine-tuning and comparing transformer models for Amharic NER, model interpretability, and vendor analytics for micro-lending. --- ## 🗂️ Project Structure ```bash ├── .github/ │ └── workflows/ # CI/CD & model evaluation workflows ├── configs/ │ └── scraping_config.yaml # Telegram API config (IDs, target channels) ├── data/ │ ├── raw/ # Raw scraped messages & images │ ├── processed/ │ │ └── cleaned.parquet # Preprocessed data (OCR + text) │ └── labeled/ │ ├── raw_for_annotation/ │ │ └── messages_for_manual_labeling.json │ ├── v1/ │ │ ├── train.conll │ │ ├── val.conll │ │ └── test.conll │ └── README.md # Labeling rules and entity definitions ├── models/ │ ├── *_finetuned/ # Fine-tuned model checkpoints (ignored by git) │ └── model_cards/ # Evaluation results, model cards ├── reports/ │ └── vendor_scorecard.csv # Vendor analytics output ├── src/ │ ├── analytics/ │ │ └── vendor_scoring.py # Vendor analytics & scorecard │ ├── data_pipeline/ │ │ ├── scraper.py # Telegram scraper │ │ ├── preprocessor.py # OCR & cleaning logic │ │ └── labeling_tool.py # Sampling & CoNLL conversion │ ├── ml_pipeline/ │ │ ├── training.py # Model …