abeni505/week4-amharic-ecommerce-extractor

# Amharic E-commerce Data Extractor for FinTech Analysis This project, part of the 10Academy B5W4 challenge, focuses on building an end-to-end data pipeline to extract structured information from unstructured Amharic e-commerce posts on Telegram. The ultimate goal is to create a FinTech engine that can assess vendor activity to identify suitable candidates for micro-lending, based on a project for a hypothetical company, **EthioMart**. ## 🚀 Key Objectives - **Data Ingestion:** Programmatically scrape and collect posts from multiple Ethiopian e-commerce Telegram channels. - **Data Annotation:** Create a high-quality, manually labeled dataset for Named Entity Recognition (NER) using the CoNLL format. - **Model Fine-Tuning:** Fine-tune a pre-trained transformer model (e.g., XLM-Roberta) to accurately identify `PRODUCT`, `PRICE`, and `LOCATION` entities in Amharic text. - **Model Evaluation & Selection:** Compare the performance of different models to choose the most suitable one for the task. - **FinTech Analytics:** Develop a "Vendor Scorecard" by combining NER-extracted entities with post metadata (like views and post frequency) to create a "Lending Score". ## 🛠️ Tech Stack - **Programming Language:** Python 3.10+ - **Data Ingestion:** Telethon - **Data Manipulation:** Pandas - **NLP/ML Framework:** Hugging Face (Transformers, Datasets, Evaluate) - **Interpretability:** SHAP - **Environment Management:** venv - **Development Environment:** JupyterLab ## 📂 Project Structure ``` week4-amharic-ecommerce-extractor/ ├── data/ │ └── labeled_data.conll # Manually labeled data for NER training ├── models/ # Saved fine-tuned models (ignored by git) ├── notebooks/ │ ├── 01_Data_Ingestion_and_Preprocessing.ipynb │ ├── 02_Data_Labeling.ipynb │ ├── 03_Model_Finetuning.ipynb │ ├── 04_Model_Comparison.ipynb │ ├── 05_Model_Interpretability.ipynb │ └── 06_FinTech_Vendor_Scorecard.ipynb ├── reports/ │ ├── …