Naod-Mergiya/Amharic-E-commerce-Data-Extractor

# Amharic E-commerce Data Extractor This project ingests raw messages from Ethiopian Telegram e-commerce channels, cleans & normalises Amharic text, and structures the output so that downstream models (e.g. entity-extractors or LLM fine-tuning pipelines) can discover seller insights. ```text ├── fetch/ # CLI entry-points & quick scripts ├── src/ # Importable Python modules │ ├── core/ # Telegram fetching logic │ ├── utils/ # Shared helpers (text prep, IO) │ ├──services │ └── config.py # Paths & environment variables ├── notebooks/ # Exploratory notebooks ├── scriptss/ # Executable scripts ├── data/ │ ├── raw/ # Appended JSONL straight from Telegram │ └── processed/ # Tokenised / cleaned records ├── requirements.txt └── .env.example # Fill with your Telegram API creds ``` ## Environment setup 1. Clone the repo and create a virtual environment (recommended) ```bash git clone github.com cd Amharic-E-commerce-Data-Extractor python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate ``` 2. Install runtime requirements ```bash pip install -r requirements.txt ``` Extra tooling used in notebooks can be installed at any time: ```bash pip install regex conllu jupyter ``` 3. Obtain Telegram API credentials • Go to and create an application. • Copy `api_id` and `api_hash`. 4. Configure credentials ```bash cp .env.example .env # then edit .env and fill TELEGRAM_API_ID, TELEGRAM_API_HASH, (optional) PHONE_NUMBER ``` ## Quick start 1. Install deps ```bash pip install -r requirements.txt ``` 2. Copy `.env.example` → `.env` and fill `TELEGRAM_API_ID` & `TELEGRAM_API_HASH` (grab from my.telegram.org). 3. Run the ingestor with at least five channel handles: ```bash python fetch/run_ingestion.py channel1 channel2 channel3 channel4 channel5 ``` Messages are appended to `data/raw/ .jsonl` as they arriv …