benkenlin/KAIM_W04_EthioMart_Amharic_Named_Entity_Recognition_System

The project aims to develop an Amharic NER (Named Entity Recognition) system to extract vital business entities (Product, Price, Location) from diverse Telegram channel content. This extracted, structured data will feed into EthioMart's centralized database, facilitating a seamless e-commerce experience for customers and vendors. # KAIM_W04_EthioMart_Amharic_Named_Entity_Recognition_System The project aims to develop an Amharic NER (Named Entity Recognition) system to extract vital business entities (Product, Price, Location) from diverse Telegram channel content. This extracted, structured data will feed into EthioMart's centralized database, facilitating a seamless e-commerce experience for customers and vendors. # EthioMart Amharic Named Entity Recognition System This project aims to develop an Amharic Named Entity Recognition (NER) system to extract key business entities (Product, Price, Location, Material, Delivery Fee, Contact Info) from Telegram e-commerce channels in Ethiopia. This will centralize data for EthioMart's platform. ## Project Structure ````` KAIM_W04_EthioMart_Amharic_Named_Entity_Recognition_System/ ├── data/ │ ├── raw/ │ │ ├── images/ │ │ ├── documents/ │ │ ├── telegram_data.csv # Output from scraper │ │ ├── telegram_messages.jsonl # Converted from CSV for consistency │ │ └── channels_to_crawl.xlsx │ ├── processed/ │ │ ├── processed_messages.jsonl # After OCR, normalization, and cleaning │ │ └── ocr_output_logs/ │ ├── labeled/ │ │ ├── train.conll │ │ ├── dev.conll │ │ └── test.conll │ └── external/ │ ├── amharic_news_ner_dataset/ │ └── shageronlinestore_sample/ ├── notebooks/ │ ├── 01_data_ingestion_preprocessing.ipynb │ ├── 02_data_labeling_guidelines.ipynb # You'd fill this with your labeling process/rules │ ├── 03_model_finetuning.ipynb │ ├── 04_model_comparison.ipynb │ ├── 05_model_interpretability.ipynb │ └── 06_vendor_scorecard_analytics.ipynb ├── scripts/ │ ├── telegram_scraper.py # Completed as per last interaction │ ├── data_preprocessing.py # Handles CSV to JSONL, OCR, text cleaning │ ├── data_splitting.py # To split your labeled CoNLL data │ ├── ner_model_training.py # Main script for fine-tuning │ ├── ner_evaluation …