Ermi-HST/EthioMart_NER_Amharic

# EthioMart Amharic Named Entity Recognition (NER) System ## Project Overview The **EthioMart Amharic Named Entity Recognition (NER)** project aims to extract key business entities such as product names, prices, and locations from unstructured text data in Amharic language Telegram messages. With the increasing use of Telegram for e-commerce activities in Ethiopia, this project provides a centralized solution for processing and analyzing product-related information from multiple independent channels. ## Objectives - **Real-time Data Extraction**: Develop a system that collects and processes messages from various Ethiopian-based e-commerce Telegram channels. - **Entity Recognition**: Fine-tune a pre-trained language model to accurately identify and extract entities like product names, prices, and locations. - **Interpretability**: Implement model interpretability techniques to ensure the predictions are transparent and understandable. ## Table of Contents 1. Installation 2. Usage 3. Project Structure 4. Model Fine-Tuning 5. Evaluation Metrics 6. License 7. Contributing 8. Acknowledgements ## Installation To run this project, you will need Python 3.6 or higher. Install the necessary libraries by executing the following command: bash pip install -r requirements.txt ## Usage The project consists of several scripts for different stages of the pipeline: Data Ingestion: Collect data from Telegram channels. bash python scripts/data_ingestion.py Data Preprocessing: Clean and prepare the raw data for labeling. bash python scripts/preprocess.py Automated Data Labeling: Generate labeled data in CoNLL format for NER. bash python scripts/auto_labeling.py Model Fine-Tuning: Fine-tune a pre-trained model on the labeled dataset. bash python scripts/model_finetuning.py ## Project Structure The project follows a structured organization to enhance readability and maintainability: bash /EthioMart_NER_Amharic /data /raw # Raw Telegram messages /processed …