duleab/Amharic-Ecommerce-Extractor

# Amharic E-commerce Data Extractor A Named Entity Recognition (NER) system for extracting product information from Amharic Telegram e-commerce channels. ## Project Overview This project aims to develop a fine-tuned NER model for Amharic language that can extract key business entities such as product names, prices, and locations from Telegram e-commerce channels. The extracted data will be used to populate EthioMart's centralized database, making it a comprehensive e-commerce hub. ## Key Objectives 1. Develop a repeatable workflow for data ingestion from Telegram channels 2. Fine-tune transformer-based models for Amharic NER 3. Compare multiple model approaches and select the best performing one 4. Apply model interpretability techniques using SHAP/LIME 5. Create a vendor scoring system for micro-lending decisions ## Project Structure ``` amharic-ecommerce-extractor/ ├── data/ # Data directory │ ├── raw/ # Raw scraped data from Telegram │ ├── processed/ # Preprocessed data │ ├── labeled/ # CoNLL formatted labeled data │ └── models/ # Saved model checkpoints ├── notebooks/ # Jupyter notebooks for each step │ ├── 01_data_collection.ipynb # Telegram data scraping │ ├── 02_data_preprocessing.ipynb # Data cleaning and normalization │ ├── 03_data_labeling.ipynb # Entity labeling and validation ├── src/ # Source code │ ├── data/ # Data collection and processing │ ├── models/ # Model training and evaluation │ └── scoring/ # Vendor scoring system ├── reports/ # Project reports │ ├── interim/ # Interim submission │ └── final/ # Final submission ├── requirements.txt # Project dependencies └── README.md # Project documentation ``` ## Current Progress We have set up the project structure and implemented the following components: 1. …