NMsby/ai-weather-etl-pipeline

Weather Data Pipeline: Complete ETL system with Apache Airflow orchestration and natural language database querying. Extract weather data from APIs, transform with PostgreSQL, and interact using conversational AI in multiple languages (English/Kiswahili). Built with Docker, Streamlit, and OpenAI integration. Overview ======== Welcome to Astronomer! This project was generated after you ran 'astro dev init' using the Astronomer CLI. This readme describes the contents of the project, as well as how to run Apache Airflow on your local machine. Project Contents ================ Your Astro project contains the following files and folders: - dags: This folder contains the Python files for your Airflow DAGs. By default, this directory includes one example DAG: - `example_astronauts`: This DAG shows a simple ETL pipeline example that queries the list of astronauts currently in space from the Open Notify API and prints a statement for each astronaut. The DAG uses the TaskFlow API to define tasks in Python, and dynamic task mapping to dynamically print a statement for each astronaut. For more on how this DAG works, see our Getting started tutorial. - Dockerfile: This file contains a versioned Astro Runtime Docker image that provides a differentiated Airflow experience. If you want to execute other commands or overrides at runtime, specify them here. - include: This folder contains any additional files that you want to include as part of your project. It is empty by default. - packages.txt: Install OS-level packages needed for your project by adding them to this file. It is empty by default. - requirements.txt: Install Python packages needed for your project by adding them to this file. It is empty by default. - plugins: Add custom or community plugins for your project to this file. It is empty by default. - airflow_settings.yaml: Use this local-only file to specify Airflow Connections, Variables, and Pools instead of entering them in the Airflow UI as you develop DAGs in this project. Deploy Your Project Locally =========================== Start Airflow on your local machine by running 'astro dev start'. This command will spin up five Docker containers on your machine, each for a different Airflow component: - Postgres: Airflow's Metadata Database - Scheduler: The Airflow …