SaraFedlu/Ethiopian-Medical-Data-Warehouse

A scalable data warehouse for collecting, cleaning, and analyzing Ethiopian medical business data scraped from Telegram channels, with integrated object detection using YOLO. # Ethiopian Medical Data Warehouse This project scrapes Ethiopian medical business data from **Telegram channels**, cleans and processes the data, stores it in a **data warehouse**, and applies **object detection using YOLO** for image analysis. ## 📌 Features - ✅ **Telegram Bot Scraper** - Extracts messages and images from public channels - ✅ **Data Cleaning & Transformation** - Processes scraped data for consistency - ✅ **Object Detection with YOLO** - Detects relevant objects in images - ✅ **Data Warehouse Integration** - Stores structured data for analysis - ✅ **Logging & Monitoring** - Tracks scraping activity --- ## 📂 Project Structure ``` 📂 ethiopian_medical_business_scraper │── 📂 data_scraper │ │── bot_scraper.py # Scrapes Telegram messages │ │── image_scraper.py # Downloads images │ │── config.py # Stores API keys │ │── logging_config.py # Logging setup │ │── 📂 data_processing │ │── clean_data.py # Cleans and transforms data │ │── process_images.py # Prepares images for YOLO │ │── 📂 object_detection │ │── yolo_detection.py # YOLO object detection │ │── model_config.py # YOLO settings │ │── 📂 data_warehouse │ │── database_setup.py # Sets up database │ │── insert_data.py # Inserts cleaned data into DB │ │── 📂 utils │ │── helpers.py # Utility functions │ │── constants.py # Stores channel names │ │── 📂 logs │ │── scraper.log # Logs scraping activity │ │── 📂 data │ │── raw/ │ │ │── messages.json # Raw scraped messages │ │ │── images/ # Downloaded images │ │── cleaned/ │ │ │── cleaned_data.csv # Processed data │ │── requirements.txt # Dependencies │── README.md # Project documentation │── .gitignore # Ignore unnecessary files ``` --- ## 🔧 Installation ### 1️⃣ **Clone the repository** ```bash git clone github.com cd ethiopian_medical_ …