duresaguye/Telegram-Medical-Data-ETL

This project is focuses on building a data warehouse for Ethiopian medical businesses by scraping data from Telegram channels. # 🚀 Telegram Medical Data ETL ## 📌 Project Overview This project is focuses on **building a data warehouse** for Ethiopian medical businesses by scraping data from **Telegram channels**. ### 🎯 **Key Objectives** - Develop a **data scraping and collection pipeline** for Telegram channels. - Implement **data cleaning and transformation** using ETL processes. - Perform **object detection using YOLO** on collected images. - Design and implement a **data warehouse** for structured storage. - Develop an **API using FastAPI** to expose the collected data. --- ## 🏢 Business Need Kara Solutions, a data science company, requires a **centralized data warehouse** to store and analyze Ethiopian medical business data from Telegram. The warehouse will: ✅ Improve **data accessibility and organization**. ✅ Enable **data analysis to detect trends and patterns**. ✅ Enhance **decision-making** through structured data. ✅ Allow **efficient querying and reporting**. --- ## 🛠️ Technologies Used - **Python** (Beautiful Soup, Scrapy, Selenium, Telethon) - **YOLO** (Object Detection) - **PostgreSQL** (Data Storage) - **DBT** (Data Transformation) - **FastAPI** (API Development) - **SQLAlchemy** (Database ORM) --- ## 📂 Project Structure ``` Telegram-Medical-Data-ETL/ │── api/ # API development using FastAPI │── data_processing/ # ETL and DBT transformations │── data_scraping/ # Telegram scraping scripts │── deployment/ # Deployment configurations │── docs/ # Documentation │── logs/ # Logging and monitoring │── notebooks/ # Jupyter notebooks for data exploration │── object_detection/ # YOLO object detection implementation │── photos/ # Collected images for processing │── scripts/ # Utility scripts │── setup.py # Setup configurations │── requirements.txt # Dependen …