Tekalig/EthioMart-Amharic-NER

# EthioMart: Telegram E-Commerce Centralization with Amharic NER ## Overview EthioMart aims to centralize e-commerce activities on Telegram by consolidating data from multiple Ethiopian-based Telegram channels into one unified platform. This project focuses on fine-tuning large language models (LLMs) for Amharic Named Entity Recognition (NER) to extract key business entities like product names, prices, and locations from text, images, and documents shared across Telegram channels. ## Business Need Telegram is increasingly popular for business transactions in Ethiopia, but its decentralized nature makes it difficult for vendors and customers to navigate multiple channels. EthioMart solves this problem by creating a centralized platform that aggregates and processes real-time data, enabling users to explore and interact with multiple vendors in one place. ## Key Objectives - Real-time data extraction from Telegram channels. - Fine-tuning LLMs to extract entities such as product names, prices, and locations. - Creating a comprehensive e-commerce hub for Ethiopian businesses. ## Possible Entities - **Product Names or Types** - **Material or Ingredients** - **Location Mentions** - **Monetary Values or Prices** ## Data - **Source**: Messages and data from Ethiopian-based Telegram e-commerce channels. - **Types**: - Text (Amharic language messages) - Images (Product images, marketing materials) ### Example Channels - ShagerOnlineStore - AddisMart - EthiopiaDeals - BoleMarket - AbayOnline ## Project Structure ```plaintext EthioMart-Amharic-NER/ ├── data/ # Raw and preprocessed data ├── src/ # Source code │ ├── data_ingestion.py # Scraper for Telegram channels │ ├── preprocessing.py # Data preprocessing functions │ ├── fine_tune_ner.py # Code for fine-tuning NER models │ ├── model_comparison.py # Model evaluation and comparison │ ├── interpretability.py # Model interpretability tools ├── notebooks/ # Jupyter n …