YitayalK/amharic-ecomm-entity-extraction

# Amharic Named Entity Recognition (NER) for EthioMart ## Project Overview EthioMart is building a centralized e-commerce platform that consolidates product listings from multiple Telegram channels in Ethiopia. This project focuses on fine-tuning a Named Entity Recognition (NER) model to extract key business entities such as product names, prices, and locations from Amharic-language messages shared on these Telegram channels. ## Features - **Real-time Data Extraction:** Fetch messages from Ethiopian e-commerce Telegram channels. - **NER Model Fine-Tuning:** Train models to identify and extract entities. - **Model Comparison:** Evaluate multiple models (XLM-Roberta, BERT-Tiny-Amharic, AfroXMLR). - **Model Interpretability:** Use SHAP and LIME to explain model predictions. - **Business Intelligence:** Provide structured e-commerce data for EthioMart. ## Project Structure ``` ├── data/ # Raw and preprocessed datasets ├── notebooks/ # Jupyter notebooks for analysis and training ├── scripts/ # Python scripts for data ingestion and model training ├── results/ # Evaluation results and reports ├── README.md # Project documentation └── requirements.txt # Required dependencies ``` ## Installation ### Prerequisites - Python 3.8+ - Jupyter Notebook / Google Colab - Telegram API access ### Setup Instructions 1. Clone the repository: ```bash git clone github.com cd amharic-ecomm-entity-extraction ``` 2. Install dependencies: ```bash pip install -r requirements.txt ``` ## Data Collection & Preprocessing 1. Use a Telegram scraper to fetch messages from selected e-commerce channels. 2. Preprocess text (tokenization, normalization, stopword removal, etc.). 3. Label a subset of messages in CoNLL format for training. ## Model Training & Fine-Tuning 1. Load pre-trained models (XLM-Roberta, BERT-Tiny-Amharic, AfroXMLR). 2. Fine-tune using the labeled Amharic dataset. 3. E …