ZekiKobe/ethiopian-medical-data-platform

# Ethiopian Medical Data Platform An end-to-end data platform that extracts medical business insights from public Telegram channels in Ethiopia through a modern ELT pipeline with object detection and analytical API. ## Table of Contents - Features - Business Insights - Technical Stack - Project Structure - Installation - Task Details - Task 1: Data Scraping - Task 2: Data Modeling - Task 3: Object Detection - Task 4: Analytical API - Task 5: Orchestration - API Documentation - Usage Examples - Contributing - License ## 🚀 Features - Telegram data scraping with Telethon - Data lake with partitioned JSON storage - Star schema data warehouse in PostgreSQL - dbt transformations with testing - YOLOv8 image object detection - FastAPI analytical endpoints - Dagster pipeline orchestration - Dockerized environment ## 📊 Business Insights Answers key questions: 1. Top 10 mentioned medical products 2. Price/availability across channels 3. Visual content analysis (pills vs creams) 4. Daily/weekly posting trends 5. Channel comparison metrics ## 🛠️ Technical Stack | Component | Technology | |--------------------|--------------------------| | Data Extraction | Telethon | | Data Storage | PostgreSQL 13 | | Transformation | dbt-core 1.10+ | | Object Detection | YOLOv8 (Ultralytics) | | API Framework | FastAPI + Uvicorn | | Orchestration | Dagster | | Infrastructure | Docker + Docker Compose | ## 🏗️ Project Structure ├── data/ # Data lake storage │ ├── raw/ # Raw scraped data │ └── processed/ # Processed data outputs ├── dbt/ # Data transformation │ └── medical_analytics/ # dbt project ├── app/ # Application code │ ├── api/ # FastAPI implementation │ └── object_detection/ # YOLO image processing ├── scripts/ # Utility scripts │ ├── scraping/ # Telegram scrapers │ └── database/ # DB management ├── docs/ # Documentation ├── tests/ # Test suites ├── docker-compose. …