mebsahle/amharic-ecommerce-ner

# 10Academy-Kifiya-Week-5 # EthioMart: Building a Named Entity Recognition (NER) System for Amharic: Challenges, Strategies, and Insights ## Introduction With the growing adoption of Telegram as a platform for e-commerce activities in Ethiopia, numerous independent channels facilitate business transactions. However, this decentralization poses significant challenges for both customers and vendors. The EthioMart NER project aims to create a centralized platform for Telegram-based e-commerce in Ethiopia by extracting key business entities (e.g., product names, prices, and locations) from Telegram messages. EthioMart’s vision is to centralize these activities, providing a unified platform that consolidates real-time data from these channels. A crucial part of this initiative involves fine-tuning Named Entity Recognition (NER) models tailored for Amharic to extract key business entities from text shared across these channels. This report outlines the step-by-step process of developing an Amharic NER system, including the collection, preprocessing, annotation, and modeling phases, as well as insights derived from each stage. The discussion concludes with key results, challenges, and recommendations for future improvements. --- ## **Data Preparation** ### **Data Ingestion** - **Objective:** Fetch messages from Ethiopian Telegram e-commerce channels. - **Steps Completed:** 1. Identified and connected to 5 Telegram channels (e.g., Shageronlinestore, ShegerMart, AddisMall). 2. Developed a Python script (`scraper.py`) using the `telethon` library to scrape messages in real-time. 3. Fetched and stored raw messages, including text and metadata (e.g., sender, timestamp, channel name). ### **Data Preprocessing** - **Objective:** Clean and normalize the raw data for further analysis. - **Steps Completed:** 1. Cleaned text data by removing special characters, emojis, and unnecessary symbols. 2. Normalized Amharic text (e.g., removed diacritics, standardized characters). 3. …