masrialx/Ethiopian-Telegram-Entity-Recognition

# 📌 **Amharic Telegram NER - Entity Extraction from E-commerce Channels** ## 🚀 **Project Overview** This project aims to extract important entities (products, prices, locations) from Ethiopian Telegram e-commerce channels. Using **Natural Language Processing (NLP)** and **Named Entity Recognition (NER)**, we will fine-tune a model to process Amharic text efficiently. --- ## 🛠 **Project Structure** ``` ├── .vscode/ # VS Code settings ├── .github/workflows/ # CI/CD workflows │ ├── unittests.yml ├── src/ # Source code │ ├── data_ingestion.py # Fetches messages from Telegram channels │ ├── data_preprocessing.py # Cleans and prepares raw data │ ├── train_ner_model.py # Fine-tunes the NER model ├── notebooks/ # Jupyter notebooks for analysis │ ├── labeling_data.ipynb # Manually labeling dataset in CoNLL format │ ├── model_comparison.ipynb # Comparing multiple NER models │ ├── model_interpretability.ipynb # Explainability analysis (SHAP & LIME) ├── data/ │ ├── labeled_dataset.conll # Manually labeled dataset for training ├── tests/ # Unit tests ├── scripts/ # Additional scripts ├── requirements.txt # Dependencies ├── README.md # Project documentation (this file) └── .gitignore # Ignore unnecessary files ``` --- ## 📌 **Project Tasks & Branches** | Task # | Task Name | Branch | Description | |--------|----------------------|-----------|-------------| | **1** | Data Ingestion & Preprocessing | `task-1` | Scrapes messages from Telegram, preprocesses text, and stores structured data. | | **2** | Dataset Labeling (CoNLL Format) | `task-2` | Manually labels a dataset with product, price, and location entities. | | **3** | Fine-Tuning the NER Model | `task-3` | Trains a Named Entity Recognition (NER) model on Amharic text. | | **4** | Model Comparison & Selection | `task-4` | Evaluates d …