Tewodros-agegnehu/Amharic-Ecom-NER

# EthioMart: Centralized Telegram E-Commerce Hub ## Project Description EthioMart envisions becoming the leading centralized hub for Telegram-based e-commerce activities in Ethiopia. With the growing reliance on Telegram for business transactions, numerous independent channels have surfaced, leading to challenges in managing product discovery, order placement, and communication across platforms. This project addresses these challenges by: - **Centralizing e-commerce data**: Aggregating real-time data from various Telegram channels into a unified platform. - **Improving user experience**: Providing customers with seamless access to diverse vendors in one location. - **Extracting key business data**: Developing a fine-tuned Amharic Named Entity Recognition (NER) system to identify essential entities such as product names, prices, and locations from textual and visual content shared on Telegram. --- ## Key Objectives ### Project Goals: 1. **Real-Time Data Collection**: Extract data from Ethiopian Telegram e-commerce channels in real time. 2. **Advanced NER Model Development**: Tailor models to identify entities including: - Product Names or Types - Materials or Ingredients - Locations - Monetary Values or Prices --- ## Tasks Overview ### Task 1: Data Ingestion and Preprocessing **Objective:** Build a robust system to fetch and preprocess data from multiple Telegram e-commerce channels. **Steps:** 1. Identify at least five relevant Telegram channels for data extraction. 2. Develop a scraper to collect messages, images, and documents in real time. 3. Preprocess text data by: - Tokenizing - Normalizing - Addressing Amharic-specific linguistic features 4. Clean and organize data into a structured format, separating metadata from message content. 5. Store preprocessed data for subsequent analysis. ### Task 2: Dataset Labeling in CoNLL Format **Objective:** Label a dataset subset in CoNLL format for NER tasks. **Entity Types:** - **B-Product**: Beginning of a …