GebSenait/medical-telegram-warehouse

Analytics warehouse that transforms Ethiopian medical Telegram data into structured insights using fraud-detection style anomaly and behavior analysis. # Medical Telegram Warehouse **Enterprise-Grade ELT Pipeline for Ethiopian Medical/Pharmaceutical Telegram Channel Data** **Organization**: Kara Solutions (Ethiopia) **Project**: `medical-telegram-warehouse` --- ## 📑 Table of Contents - Business Understanding - Architecture Overview - Task 1: Data Scraping & Collection - Implementation Details - Results & Insights - Task 2: Data Modeling & Transformation - Implementation Details - Star Schema Design Decisions - Results & Insights - Task 3: Data Enrichment with Object Detection (YOLO) - Implementation Details - Analysis & Insights - Results & Limitations - Task 4: Analytical API (FastAPI) - Implementation Details - API Endpoints - Query Logic & Analysis - Results & Insights - Task 5: Pipeline Orchestration (Dagster) - Implementation Details - Pipeline Design & DAG - Execution Steps - Monitoring Results & Insights --- ## 🎯 Business Understanding This project extracts insights from public Ethiopian medical and pharmaceutical Telegram channels, applying fintech fraud detection principles (anomaly detection, volume spikes, behavior patterns) to the medical/pharmaceutical domain. The pipeline transforms raw Telegram data into a trusted analytical warehouse, enabling insights similar to fintech fraud detection: - **Volume anomalies**: Detect unusual posting patterns - **Behavior patterns**: Identify channel-specific characteristics - **Content richness**: Analyze message quality and media presence - **Temporal trends**: Track engagement over time --- ## 🏗️ Architecture Overview ``` ┌─────────────────┐ │ Telegram API │ └────────┬────────┘ │ │ Extract ▼ ┌─────────────────┐ │ Raw Data Lake │ │ (JSON Files) │ └────────┬────────┘ │ │ Load ▼ ┌─────────────────┐ │ PostgreSQL │ │ (raw schema) │ └────────┬────────┘ │ │ Transform ▼ ┌─────────────────┐ │ dbt Layer │ │ (Star Schema) │ └────────┬────────┘ │ │ Enrich ▼ ┌─────────────────┐ │ YOLO Layer │ └────────┬────────┘ │ │ Expose ▼ ┌───────── …