EyasuGet/EthioMart-amharic-ecommerce-entity-extractor

# EthioMart Named Entity Recognition (NER) Project ## Overview EthioMart NER is designed to transform FinTech decision-making in Ethiopia by structuring decentralized e-commerce data from Telegram channels. The project addresses the challenge of inaccessible merchant data—such as product listings, prices, and sales locations—scattered across informal Telegram groups. By consolidating and analyzing this data, EthioMart enables data-driven "vendor scorecards" for micro-lending, supporting financial inclusion for small and medium e-commerce businesses. --- ## Project Structure ``` EthioMart_NER_Project/ ├── .gitignore ├── README.md ├── requirements.txt ├── config.py ├── venv/ ├── data/ │ ├── raw_telegram_data/ │ ├── preprocessed_data/ │ │ ├── all_telegram_ecommerce_messages.json │ │ └── preprocessed_amharic_ecommerce_messages.json │ ├── labeled_data/ │ │ ├── annotation_subset.json │ │ └── labeled_data.conll │ └── (optional) media/ ├── scripts/ │ ├── scraper/ │ │ └── telegram_scraper.py │ ├── preprocessing/ │ │ └── amharic_preprocessing.py │ ├── labeling_prep/ │ │ ├── extract_annotation_subset.py │ │ └── manual_conll_labeler.py │ ├── model_training/ │ │ └── train_ner_model.py │ └── analytics/ │ └── vendor_scorecard.py ├── docs/ │ ├── Interim_Report_EthioMart.pdf │ └── Annotation_Guidelines.md └── notebooks/ ├── Task3_FineTune_NER_Model.ipynb ├── Task4_Model_Comparison.ipynb ├── Task5_Model_Interpretability.ipynb └── Task6_FinTech_Vendor_Scorecard.ipynb ``` --- ## Setup Instructions 1. **Clone the Repository** ```bash git clone cd EthioMart_NER_Project ``` 2. **Create and Activate a Python Virtual Environment** ```bash python3 -m venv venv # macOS/Linux source venv/bin/activate # Windows (CMD) venv\Scripts\activate.bat # Windows (PowerShell) .\venv\Scripts\Activate.ps1 ``` 3. **Install Dependencies** ```bash pip install -r requirements.txt ``` - If you encounter `ReadTimeoutError` (e.g., with `docc …