NabloP/b5w4-amharic-ecommerce-data-extractor-challenge

Week 4 submission for the 10 Academy Amharic E-Commerce Challenge. Implements a modular NLP pipeline for extracting structured insights from Amharic Telegram posts. Features include data ingestion, CoNLL-format NER labeling, fine-tuning multilingual models for entity extraction, and vendor analytics for loan readiness scoring. # B5W4: Amharic E-Commerce Data Extractor Challenge – 10 Academy ## 🗂 Challenge Context This repository documents the submission for 10 Academy’s **B5W4: Amharic E-Commerce Data Extractor Challenge**. The goal is to support EthioMart in becoming Ethiopia’s centralized hub for Telegram-based e-commerce by: - Extracting key business entities (product, price, location) from unstructured Amharic Telegram messages - Fine-tuning transformer models for accurate Amharic NER - Scoring vendors based on their activity, reach, and pricing to enable data-driven micro-lending The project simulates the role of a fintech data analyst building a structured NLP pipeline for intelligent vendor profiling. ### Key Features - 🧲 Real-time Telegram scraping of e-commerce messages and metadata - ✍️ CoNLL-format labeling of Amharic text with Product, Price, and Location entities - 🤖 Transformer-based fine-tuning (XLM-Roberta, mBERT) for NER extraction - 📊 Vendor-level analytics and micro-lending scorecards - 🔍 Model explainability using SHAP and LIME --- ## 🔧 Project Setup ### 1. Clone the repository: git clone github.com cd b5w4-amharic-ecommerce-data-extractor-challenge ### 2. Create and activate a virtual environment: **On Windows (PowerShell):** python -m venv data-extractor-challenge data-extractor-challenge\Scripts\Activate.ps1 **On macOS/Linux:** python3 -m venv data-extractor-challenge source data-extractor-challenge/bin/activate ### 3. Install dependencies: pip install -r requirements.txt --- ## 📁 Project Structure b5w4-amharic-ecommerce-data-extractor-challenge/ ├── data/ │ ├── raw/ │ ├── cleaned/ │ ├── labeled/ │ ├── outputs/ │ └── logs/ ├── src/ │ ├── ingestion/ │ ├── preprocessing/ │ ├── labeling/ │ ├── modeling/ │ ├── evaluation/ │ └── analytics/ ├── scripts/ │ ├── ingest_data.py │ ├── label_data.py │ ├── fine_tune_model.py │ ├── evaluate_models.py │ └── generate_sco …