masrialx/Ethiopian-Telegram-Entity-Recognition
Domaine:
natural language processingType de record:
project
Créateur:
mas
HĂ´te:
# đź“Ś **Amharic Telegram NER - Entity Extraction from E-commerce Channels**
## 🚀 **Project Overview**
This project aims to extract important entities (products, prices, locations) from Ethiopian Telegram e-commerce channels. Using **Natural Language Processing (NLP)** and **Named Entity Recognition (NER)**, we will fine-tune a model to process Amharic text efficiently.
---
## đź› **Project Structure**
```
├── .vscode/ # VS Code settings
├── .github/workflows/ # CI/CD workflows
│ ├── unittests.yml
├── src/ # Source code
│ ├── data_ingestion.py # Fetches messages from Telegram channels
│ ├── data_preprocessing.py # Cleans and prepares raw data
│ ├── train_ner_model.py # Fine-tunes the NER model
├── notebooks/ # Jupyter notebooks for analysis
│ ├── labeling_data.ipynb # Manually labeling dataset in CoNLL format
│ ├── model_comparison.ipynb # Comparing multiple NER models
│ ├── model_interpretability.ipynb # Explainability analysis (SHAP & LIME)
├── data/
│ ├── labeled_dataset.conll # Manually labeled dataset for training
├── tests/ # Unit tests
├── scripts/ # Additional scripts
├── requirements.txt # Dependencies
├── README.md # Project documentation (this file)
└── .gitignore # Ignore unnecessary files
```
---
## đź“Ś **Project Tasks & Branches**
| Task # | Task Name | Branch | Description |
|--------|----------------------|-----------|-------------|
| **1** | Data Ingestion & Preprocessing | `task-1` | Scrapes messages from Telegram, preprocesses text, and stores structured data. |
| **2** | Dataset Labeling (CoNLL Format) | `task-2` | Manually labels a dataset with product, price, and location entities. |
| **3** | Fine-Tuning the NER Model | `task-3` | Trains a Named Entity Recognition (NER) model on Amharic text. |
| **4** | Model Comparison & Selection | `task-4` | Evaluates d …