Bereket-07/Telegram-Ethiopian-Medical-channel-data-warehouse-

This repository contains ETL and ELT frameworks designed for data scraping from Telegram channels, transforming the data, and loading it into a data warehouse. It also includes YOLO-based image processing for photos, a REST API with CRUD operations, and a simple, engaging front-end interface. # 🧑‍⚕️ Telegram-Ethiopian-Medical-channel-data-warehouse ## Table of Contents - Overview - Technologies - Folder Organization - Setup - Notes - Contributing - License ## Overview: Key Functionalities ## 1. Project Overview In this project, I developed a medical data pipeline by scraping data from a Telegram channel using Telethon and storing it in a PostgreSQL database. I also extracted images from the channel and applied YOLO for object detection to generate labels. Following an ETL process, the labeled data was stored in a vector database. Finally, I implemented CRUD operations using FastAPI and built a frontend to facilitate user interaction with the data. ## 2. Data Scraping Functionality - **Utilized Telethon to scrape medical data from a Telegram channel.** - **Stored the scraped data in a PostgreSQL database** - **Extracted additional information from the channel for enhanced data quality.** ## 3. Object Detection - **Implemented YOLO (You Only Look Once) for object detection in the collected medical images.** - **Preprocessed images to prepare them for analysis** - **Labeled identified objects within images, facilitating better data interpretation and insights** ## 4. FastAPI Endpoint - **Developed a FastAPI application to create, read, update, and delete (CRUD) medical data entries** - **Enabled seamless interaction with the PostgreSQL database for managing the scraped data and object detection results.** ## 6. Frontend Development - **Created a user-friendly frontend interface for interacting with the FastAPI backend.** - **Allowed users to visualize scraped data, view object detection results, and manage entries through a simple and intuitive interface.** ## 7. Conclusion **This project effectively demonstrates the integration of various technologies to create a comprehensive medical data processing system. By leveraging Telethon for data scraping, we were able to gather valuable insights from a Telegram channel and store them in a PostgreSQL …