nebiyou27/EthioMart_Telegram_NER

A project to fine-tune LLMs for Amharic NER, extracting entities from Telegram e-commerce channels # **EthioMart Telegram NER** ## **Overview** EthioMart aims to revolutionize the Ethiopian e-commerce ecosystem by creating a centralized platform for Telegram-based e-commerce activities. This project focuses on fine-tuning large language models (LLMs) to perform **Named Entity Recognition (NER)** on Amharic text data extracted from Telegram channels. The extracted entities will populate EthioMart's centralized database, making it easier for users to explore products, prices, and locations in one unified platform. --- ## **Business Need** Telegram is a popular platform for e-commerce in Ethiopia, with numerous independent channels operating in isolation. Customers and vendors face challenges due to the lack of a unified system for product discovery and transaction management. EthioMart addresses this gap by consolidating real-time data from multiple channels into a single platform, enabling seamless interaction between vendors and customers. --- ## **Key Objectives** 1. **Data Ingestion**: Extract real-time messages, images, and documents from Telegram e-commerce channels. 2. **Preprocessing**: Tokenize, normalize, and handle Amharic-specific linguistic features for text data. 3. **Named Entity Recognition**: Fine-tune LLMs to extract entities such as: - Product Names - Prices - Locations 4. **Model Comparison**: Evaluate and compare the performance of different NER models. 5. **Business Intelligence**: Use extracted entities to enhance the user experience and business insights. --- ## **Folder Structure** ```plaintext EthioMart_Telegram_NER/ ├── .github/ │ └── workflows/ # GitHub Actions workflows for CI/CD ├── data/ │ ├── raw/ # Raw data from Telegram channels │ └── preprocessed/ # Preprocessed data ready for analysis ├── scripts/ # Python scripts for data ingestion and preprocessing ├── models/ # Saved models and checkpoints ├── notebooks/ # Jupyter notebooks for experimentati …