mihretgold/Ethiopian-Medical-Business-Data-Ware-house

# Ethiopian Medical Business Data Warehouse ## Overview This project involves building a data warehouse to store and analyze data on Ethiopian medical businesses scraped from Telegram channels. The project includes developing a data scraping and collection pipeline, cleaning and transforming the data, integrating object detection using YOLO, and designing and implementing the data warehouse. ## Table of Contents - Business Need - Project Structure - Technologies Used - Setup and Installation - Data Scraping and Collection - Data Cleaning and Transformation - Object Detection using YOLO - API Development with FastAPI - Learning Outcomes - Competency Mapping - References ## Business Need Kara Solutions, a leading data science company, requires a robust and scalable data warehouse to store and analyze data on Ethiopian medical businesses. The centralized data storage allows comprehensive analysis to uncover valuable insights, enabling better decision-making and efficient querying and reporting. ## Project Structure ```plaintext my_project/ ├── main.py ├── database.py ├── models.py ├── schemas.py └── crud.py ``` ## Technologies Used - Python - BeautifulSoup - Scrapy - Selenium - YOLO (You Only Look Once) - PostgreSQL - FastAPI - DBT (Data Build Tool) - SQLAlchemy - Pydantic ## Setup and Installation 1. **Clone the repository:** ```sh git clone github.com cd your-repo-name ``` 2. **Install dependencies:** ```sh pip install -r requirements.txt ``` 3. **Set up the database:** Configure your database connection in `database.py`. ## Data Scraping and Collection 1. **Telegram Scraping:** - Use the Telegram API or custom scripts to extract data from public Telegram channels. - Example channels: - DoctorsET - Chemed Telegram Channel - EAHCI 2. **Image and Data Scraping:** - Collect images from channels like Chemed Telegram Channel for object detection. - Use Python packages like `telethon` for extraction. ## Data Cleanin …