Mewael-EME/Healthcare_insights_telegram_yolo_dbt_api_week_07
Domaine:
healthcareType de record:
software
Créateur:
Mew
Hôte:
End-to-end data pipeline for Ethiopian medical businesses using Telegram data, dbt, YOLOv8, and FastAPI – Week 07 challenge
# 🧠 Healthcare Insights Telegram Project
An end-to-end data platform that scrapes medical-related messages from public Telegram channels, transforms the data into a clean star schema using DBT, enriches it with object detection via YOLOv8, exposes analytics via FastAPI, and orchestrates it all using Dagster.
---
## 📦 1. Setup Instructions
### 🔐 Environment Variables
Create a `.env` file in the project root:
```env
API_ID=your_api_id
API_HASH=your_api_hash
PHONE_NUMBER=+251XXXXXXX
PG_HOST=localhost
PG_PORT=5432
PG_USER=postgres
PG_PASS=2328
PG_DB=telegram_medical
API_KEY=your_secure_api_key
```
> ⚠️ Never commit your `.env` file to version control.
---
### 📦 Install Dependencies
```bash
pip install -r requirements.txt
```
If not using `requirements.txt`:
```bash
pip install fastapi uvicorn dbt psycopg2-binary python-dotenv ultralytics dagster dagster-webserver
```
---
## 🛠️ 2. Running the Pipeline
### 📥 Step 1: Scrape Telegram Data
```bash
python scrapers/scrape_telegram.py
```
### 💾 Step 2: Load JSON into PostgreSQL
```bash
python loaders/load_raw_to_postgres.py
```
### 🧠 Step 3: Run YOLOv8 Object Detection (Optional for Task 3)
```bash
python enrichment/run_yolo_inference.py
```
### 🔄 Step 4: Run DBT Models
```bash
cd medical_dbt
dbt run # Build all models
dbt test # Run tests
dbt docs generate && dbt docs serve # Open DBT docs
```
---
## 📊 3. Analytical API with FastAPI
### 🔥 Run the API server
```bash
cd api
uvicorn main:app --reload
```
### 🔐 API Key Security
Pass `?api_key=your_secure_api_key` in every request.
### 🧪 Example Endpoints
| Endpoint | Description |
|----------|-------------|
| `/api/reports/top-products?limit=10&api_key=...` | Most frequently mentioned products |
| `/api/channels/{channel_name}/activity?api_key=...` | Posting trends per channel |
| `/api/search/messages?query=paracetamol&api_key=...` | Search messages by keyword |
### 📄 Swagger UI
Visit: 127.0.0.1
---
## 🛠️ 4. Dags …