nardos-tsige/medical-telegram-warehouse

End-to-end data pipeline for Ethiopian medical business analytics using Telegram data, dbt, YOLOv8, FastAPI, and Dagster. # 🏥 Medical Telegram Warehouse An end-to-end data pipeline for analyzing Ethiopian medical businesses from public Telegram channels. --- ## 📋 Table of Contents - Overview - Business Questions - Architecture - Tech Stack - Project Structure - Quick Start - Environment Variables - Tasks - API Endpoints - Results - Testing - CI/CD Pipeline - Docker Support - Monitoring - Deployment - Branch Strategy - Troubleshooting - Contributing - License - Acknowledgments - Contact --- ## 📖 Overview This project builds a robust data platform that generates actionable insights about Ethiopian medical businesses using data scraped from public Telegram channels. The platform implements a modern ELT (Extract, Load, Transform) framework with a layered data architecture. ### Business Need A well-designed data platform significantly enhances data analysis. This platform answers key business questions such as: - What are the top 10 most frequently mentioned medical products or drugs across all channels? - How does the price or availability of a specific product vary across different channels? - Which channels have the most visual content (e.g., images of pills vs. creams)? - What are the daily and weekly trends in posting volume for health-related topics? --- ## 🎯 Business Questions | Question | How It's Answered | |---|---| | Top 10 mentioned products | API: `/api/reports/top-products` | | Product price/availability across channels | API: `/api/channels/{channel_name}/activity` | | Channels with most visual content | API: `/api/reports/visual-content` | | Daily/weekly posting trends | API: `/api/channels/{channel_name}/activity` | --- ## 🏗️ Architecture ┌─────────────────────────────────────────────────────────────────────────────┐ │ MEDICAL TELEGRAM WAREHOUSE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ …