projwork/amharic-ecommerce-extractor

# Amharic E-commerce Data Extractor A comprehensive data ingestion and preprocessing system for Ethiopian Telegram e-commerce channels. This project scrapes, processes, and analyzes messages from multiple Ethiopian e-commerce channels with support for Amharic text processing. ## 🎯 Features - **Multi-channel scraping**: Automated data collection from 6+ Ethiopian e-commerce channels - **Amharic text processing**: Specialized preprocessing for Amharic language content - **Rate limiting**: Respects Telegram API limits with intelligent rate limiting - **Entity extraction**: Extracts prices, contact information, and product details - **Data quality assessment**: Comprehensive quality reports and metrics - **Structured storage**: Organized data storage in JSON and CSV formats - **Jupyter notebook**: Interactive analysis and visualization capabilities ## 📋 Ethiopian E-commerce Channels The system currently scrapes data from these channels: 1. **@sinayelj** - Sinaye LJ Store 2. **@Shewabrand** - Shewa Brand 3. **@helloomarketethiopia** - Hello Market Ethiopia 4. **@modernshoppingcenter** - Modern Shopping Center 5. **@qnashcom** - Qnash.com 6. **@Shageronlinestore** - Shager Online Store ## 🚀 Quick Start ### 1. Prerequisites - Python 3.8 or higher - Telegram API credentials (get from my.telegram.org) ### 2. Installation ```bash # Clone the repository cd amharic-ecommerce-extractor # Install dependencies pip install -r requirements.txt ``` ### 3. Configuration Create a `.env` file in the project root with your Telegram API credentials: ```env TG_API_ID=your_api_id_here TG_API_HASH=your_api_hash_here phone=+251911234567 # Your phone number with country code ``` ### 4. Run Data Ingestion **Option 1: Demo without Telegram API (for testing)** ```bash python scripts/demo_preprocessing.py ``` **Option 2: Simple scraping** ```bash python scripts/telegram_scrapper.py ``` **Option 3: Full pipeline (scraping + preprocessing)** ```bash python scripts/run_data_ing …