asiya-km/Building-an-Amharic-E-commerce-Data-Extractor

# Amharic E-commerce Data Extractor ## Overview This project collects, processes, and analyzes data from Ethiopian Telegram e-commerce channels, fine-tunes NER models for Amharic, and provides vendor analytics for micro-lending. ## Folder Structure ``` Building an Amharic E-commerce Data Extractor/ ├── data/ │ ├── raw/ # Raw Telegram data (JSON, CSV, images) │ ├── processed/ # Cleaned/tokenized data │ └── labeled/ # CoNLL-labeled files for NER ├── models/ # Saved fine-tuned models ├── scripts/ # Python scripts for each task ├── reports/ # Jupyter notebooks, analysis, and final report ├── requirements.txt # Python dependencies └── README.md ``` ## Setup 1. Install dependencies: ```bash pip install -r requirements.txt ``` 2. Create the data folders: ```bash mkdir -p data/raw data/processed data/labeled models scripts reports ``` ## Scripts - `scripts/data_ingestion.py`: Scrape Telegram channels and save raw data. - `scripts/preprocessing.py`: Clean and tokenize Amharic text. - `scripts/labeling_template.py`: Template for manual CoNLL labeling. - `scripts/train_ner.py`: Fine-tune NER models. - `scripts/compare_models.py`: Compare different NER models. - `scripts/interpretability.py`: Model interpretability with SHAP/LIME. - `scripts/vendor_scorecard.py`: Vendor analytics and scorecard. ## Data - Place raw Telegram data in `data/raw/`. - Place processed/tokenized data in `data/processed/`. - Place labeled CoNLL files in `data/labeled/`. ## Usage See each script for usage instructions. Most scripts can be run as standalone Python files or in Jupyter notebooks for interactive analysis.