senaygui/AiML-ethiopian-medical-biz-datawarehouse

The Ethiopian Medical Business Data Warehouse & Analytics Platform is a comprehensive data solution tailored to enhance the efficiency and efficacy of Ethiopia's healthcare and medical sectors. # Ethiopian Medical Data Warehouse and Analytics Pipeline The Ethiopian Medical Business Data Warehouse & Analytics Platform enhances the efficiency of Ethiopia's healthcare sector by creating a robust data warehouse. It extracts data and images from public Telegram channels related to Ethiopian medical businesses, performs object detection on the images, and cleans, transforms, and stores the data in the warehouse. The main goal is to provide a unified solution for data analysis, supporting informed decision-making and driving strategic advancements in healthcare. ### Technologies/Tools Used: Python, DBT, SQL, ETL, PostgreSQL, FastAPI, Pandas, Pytest, SQLAlchemy, YOLOv5 Postman, CI/CD, Jupiter Notebook,Git , PDF & Google Drive (for project report). ### Key Accomplishments - **ETL Process:** Successfully managed the end-to-end ETL process, including data extraction, cleaning, transformation, and loading. - **DBT for Data Modeling:** Implemented data modeling and transformation using SQL with DBT. - **Image Extraction and Object Detection:** Extracted images from Telegram channels, performed object detection, and stored the results back into the data warehouse. - **Database Management:** Loaded cleaned data into a PostgreSQL database. - **API Development:** Exposed cleaned data for analysis through APIs using FastAPI, facilitating easy access from the database/data warehouse. - **Project Documentation:** Prepared comprehensive documentation for each step of the project to ensure clarity and understanding for the client. ## Table of Contents 1. Data Scraping and Collection Pipeline 2. Data Cleaning and Transformation 3. Object Detection Using YOLO 4. Exposing the Collected Data Using FastAPI 5. Postman Collection 6. Installation 7. Usage 8. Contributing 9. License ## Data Scraping and Collection Pipeline ### Telegram Scraping Utilize the Telegram API or custom scripts to extract data from public Telegram channels related to Ethiopian medical businesses. K …