Miheret-Girmachew/Amharic-FinTech-Extractor

An Amharic NER system using fine-tuned Transformer models (BERT) to extract e-commerce data from Telegram for FinTech vendor analysis. # B5W4: Amharic E-commerce Data Extractor for FinTech Analysis An NLP system to extract structured data from Amharic e-commerce posts on Telegram and generate a "Vendor Scorecard" for FinTech analysis. --- ## 1. Overview EthioMart's vision is to become the primary hub for Telegram-based e-commerce in Ethiopia. Currently, the market is fragmented across numerous independent channels, creating a decentralized and inefficient experience for both vendors and customers. This project is the foundational phase in developing a centralized platform to solve this issue. The primary objective is to build a robust Amharic Named Entity Recognition (NER) system that can automatically ingest, process, and extract structured business information from unstructured Telegram posts. This structured data is the key to creating a "Vendor Scorecard," a FinTech tool that will help EthioMart identify promising and reliable vendors for services like micro-lending and logistics partnerships. ### Key Features - **Automated Data Ingestion:** Programmatically scrapes thousands of posts from public Amharic e-commerce Telegram channels. - **Named Entity Recognition (NER):** Identifies and extracts key business entities like PRODUCT, PRICE, and LOCATION. - **High-Quality Labeled Data:** Provides a manually annotated dataset in the standard CoNLL format, ready for model training. - **Vendor Analytics Foundation:** The extracted data powers a "Vendor Scorecard" by analyzing product types, pricing strategies, post frequency, and market reach (via post views). --- ## 2. Technology Stack - **Data Ingestion:** Python, Telethon - **Data Processing:** Pandas, NumPy - **NLP Model:** Hugging Face Transformers (for fine-tuning models like XLM-Roberta, mBERT) - **Data Annotation:** CoNLL (BIO) format - **Development Environment:** Jupyter Notebooks, Python Virtual Environments --- ## 3. Data Workflow & NER Model The project follows a multi-stage data pipeline to transform raw, unstructured te …