Nahom-ketsela/Ethiopian-Medical-Business-Data-Warehouse

# Ethiopian Medical Business Data Warehouse This project is a comprehensive data engineering solution designed to collect, clean, transform, and store data related to Ethiopian medical businesses. It scrapes data from public Telegram channels, processes it using advanced techniques like object detection with YOLO, and stores it in a centralized data warehouse. The data is exposed via a FastAPI application for easy access and analysis. --- ## 🚀 Key Features - **Automated Telegram Data Scraping**: Collects text and image data from public Telegram channels. - **Data Cleaning & Transformation**: Cleans and standardizes data for consistency and usability. - **Object Detection with YOLO**: Extracts insights from images using state-of-the-art object detection. - **Centralized Data Storage**: Stores processed data in a PostgreSQL database for efficient querying. - **API Access**: Exposes data via a FastAPI application for easy integration and retrieval. - **Scalable Design**: Built to handle large volumes of data and extendable for additional sources. - **Interactive Exploration**: Includes Jupyter Notebooks for step-by-step data analysis. --- ## 🛠 Project Structure ``` WEEK7/ │── data/ # Raw and cleaned data storage │ ├── images │ ├── raw_telegram_data.csv │ ├── yolo_detections.csv │ ├── cleaned_telegram_data.csv # Cleaned merged dataset │── logs/ # Logging directory │── FasrApi/ # FastApi for the backend │ ├── main.py │ ├── crud.py │ ├── database.py │ ├── models.py │ ├── schema.py │── Frontend/ # React.js for the frontend │── notebook/ # Jupyter Notebooks for analysis │ ├── data_cleaning.ipynb | ├── load_yolov5_to_db.ipynb | ├── Object_detection_using_yolo.ipynb │── scripts/ # Python scripts for processing │ ├── data_cleaning.py # Functions for data cleaning │ ├── database_connection.py …