Filimon-H/ethio-med-telegram-analytics
Domain:
natural language processinghealthcareRecord type:
softwareproject
Creator:
Fil
Host:
End-to-end data pipeline analyzing Ethiopian medical Telegram channels. Includes data scraping, warehouse modeling with dbt, image detection (YOLOv8), and FastAPI analytics. Built with Docker, PostgreSQL, and Dagster orchestration."
# 🏥 ETHIO-MED-TELEGRAM-ANALYTICS
**Containerized analytics platform** for Ethiopian medical Telegram data 📦🇪🇹
End-to-end pipeline for NLP/CV tasks: Named Entity Recognition (NER) & YOLOv8 image detection
---
## 🧠 Project Overview
Robust **ELT pipeline** ingesting Amharic Telegram messages from Ethiopian medical channels. Processes data for:
- 🧠 Amharic NER (Named Entity Recognition)
- 👁️ YOLOv8 medical object detection
- 📊 Business analytics API
---
## 🔧 Tech Stack
| Layer | Tool |
|-------------------|-------------------------------|
| 🐘 **Database** | PostgreSQL 14 + pgAdmin |
| 🧱 **Transform** | dbt (Data Build Tool) |
| 📤 **Ingestion** | Telethon + Custom Python |
| 🔍 **API** | FastAPI + Pydantic |
| 🤖 **ML Models** | YOLOv8, HuggingFace Transformers |
| ⚙️ **Orchestration** | Dagster |
| 🐳 **Infra** | Docker + Docker Compose |
---
## 📂 Directory Structure
├── data/ # Telegram data
│ ├── raw/ # Raw JSONs
│ └── preprocessed/ # Cleaned data
├── ethio_api/ # FastAPI app
├── ethio_pipeline/ # Dagster jobs
├── telegram_dbt_project/ # dbt models
├── scripts/ # Data scripts
├── notebooks/ # EDA/ML experiments
├── .env # Environment secrets
├── docker-compose.yml # Multi-container setup
├── requirements.txt # Dependencies
└── README.md
## 🚀 Quickstart Guide
### 1. Clone & Configure
```bash
git clone github.com
cd ETHIO-MED-TELEGRAM-ANALYTICS
cp .env.example .env
📝 Edit .env with PostgreSQL and Telegram credentials
2. Launch Containers
docker compose up --build
3. Scrape Messages
python scripts/scrape_telegram.py --channel @cheMed123 --limit 1000
4. Load Raw Data
python scripts/load_raw_json.py --input data/raw/telegram_messages/YYYY-MM-DD/
5. Run dBT Transformations
docker exec -w /app/telegram_dbt_project -it telegram_app dbt build
6. Serve Documentation (Optional)
docker exec -w /app/te …