wastb/NER-For-Amharic-E-Commerce-Data

This project focuses on building an Amharic Named Entity Recognition (NER) system to extract important business entities—such as product names, prices, and locations—from the messages shared in these Telegram channels. The extracted data will be used to populate EthioMart's centralized database. # EthioMart - Named Entity Recognition for Amharic E-commerce Data ## Overview EthioMart, a growing hub for Telegram-based e-commerce in Ethiopia, aims to consolidate multiple independent e-commerce channels into a single centralized platform. With the increasing use of Telegram for business transactions, customers and vendors are currently spread across various channels, leading to challenges in product discovery, communication, and order management. This project focuses on building an **Amharic Named Entity Recognition (NER) system** to extract important business entities—such as product names, prices, and locations—from the messages shared in these Telegram channels. The extracted data will be used to populate EthioMart's centralized database, providing a seamless and organized shopping experience for customers and a unified platform for vendors. ## Key Objectives 1. **Real-time Data Extraction**: Fetch data from various Ethiopian Telegram e-commerce channels. 2. **Fine-tuning Large Language Models (LLMs)**: Adapt existing LLMs to accurately extract business entities like product names, prices, and locations from Amharic text. ## Tasks Breakdown ### Task 1: Data Ingestion and Data Preprocessing This task involves building a system that collects messages from multiple Ethiopian Telegram e-commerce channels, processes them, and prepares the data for entity extraction. #### Steps: 1. **Identify and Connect to Relevant Telegram Channels**: - Develop a custom scraper to fetch real-time data from selected Telegram e-commerce channels. - List of relevant channels will be identified for data collection. 2. **Ingest Messages**: - Collect text messages, images, and documents from these channels. - Ensure real-time collection of data as they are posted. 3. **Preprocess Text Data**: - Tokenize and normalize Amharic text. - Handle Amharic-specific linguistic features such as script and morphology. - Structure the data into a unifie …