YonInsights/EthioMart-Amharic-NER-System-for-Telegram-E-Commerce-Integration

This project develops an Amharic Named Entity Recognition (NER) system to streamline Ethiopian e-commerce on Telegram. It uses fine-tuned Large Language Models (LLMs) to extract key business information—product names, prices, and locations—from unstructured text, images, and documents shared in Telegram channels. # EthioMart Amharic NER System for Telegram E-Commerce Integration ## Project Overview This project focuses on consolidating Ethiopian e-commerce activities conducted via Telegram by developing an Amharic Named Entity Recognition (NER) system. By leveraging fine-tuned Large Language Models (LLMs), the system extracts key business entities—product names, prices, and locations—from unstructured text, images, and documents shared in multiple Telegram channels. The extracted entities are organized into a centralized database, enabling EthioMart to provide a unified platform for customers and vendors. This project is part of an effort to revolutionize Ethiopian e-commerce by improving accessibility, convenience, and efficiency for all stakeholders. --- ## Objectives 1. **Entity Extraction**: - Extract key entities (e.g., product names, prices, and locations) from Telegram messages in the Amharic language. 2. **Model Fine-tuning**: - Fine-tune multilingual LLMs such as XLM-Roberta and BERT-tiny-Amharic for Amharic NER tasks. 3. **Model Evaluation**: - Compare the performance of multiple models and select the best-performing one. 4. **Interpretability**: - Use tools like SHAP and LIME to provide transparency into model predictions. 5. **Integration**: - Develop a pipeline for real-time data extraction and storage in a centralized database. --- ## Dataset Overview - **Source**: Messages from Ethiopian e-commerce Telegram channels. - **Types**: - **Text**: Amharic language messages. - **Images**: Product images and marketing materials. - **Metadata**: Sender details, timestamps. - **Labeling**: CoNLL format for Named Entity Recognition (NER), identifying entities such as product names, prices, and locations. --- ## Technologies Used - **Programming Language**: Python - **Libraries**: - NLP: Hugging Face Transformers, NLTK - Data Manipulation: pandas, numpy - Data Visualization: matplotlib, seaborn - Model Evaluation: scikit-learn - **Database**: MySQL for structu …