aman3013/Fine-tuning-Amharic-NER

# Fine-tuning-Amharic-NER ## Overview EthioMart aims to become the primary hub for Telegram-based e-commerce activities in Ethiopia. As Telegram's popularity for business transactions grows, various independent e-commerce channels have emerged, creating challenges for both vendors and customers. EthioMart plans to address these challenges by developing a centralized platform that consolidates real-time data from multiple Telegram channels. The primary focus of this project is to fine-tune a Large Language Model (LLM) for Amharic Named Entity Recognition (NER) to extract key business entities such as product names, prices, and locations from text, images, and documents shared across Telegram channels. ## Key Objectives - Real-time data extraction from Telegram channels - Fine-tuning LLM to extract entities such as product names, prices, and locations ### Possible Entities - **Product Names or Types** - **Material or Ingredients**: Specific mentions of materials used in the products. - **Location Mentions** - **Monetary Values or Prices** ## Data - **Source**: Messages and data from Ethiopian-based e-commerce Telegram channels. - **Types**: - Text (Amharic language messages) - Images (Product images, marketing materials) ## Knowledge and Skills - **Text Processing**: Handling Amharic text, tokenization, and preprocessing techniques. - **LLM Fine-tuning**: Adapting large language models for Amharic NER tasks. - **Model Comparison & Selection**: Evaluating performance using metrics like F1-score, precision, and recall. - **Model Interpretability**: Using tools such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) to explain model predictions and outputs. ## Learning Outcomes By the end of this challenge, participants will have: - A working pipeline for entity extraction from Amharic Telegram messages. - A performance analysis of different models and their interpretability. - Insights into how the extracted …