Pipeline-Africa/lessons

# Data Engineering Course A 12-week hands-on program covering the full data engineering stack — from SQL fundamentals to building, scheduling, and deploying production-grade data pipelines. --- ## What You Will Build By Week 12, you will have: - A working local development environment (PostgreSQL, Python, Git, Docker) - 10+ projects and labs pushed to a public GitHub portfolio - A containerised, cloud-deployed end-to-end pipeline - A Power BI dashboard connected to a data warehouse you built - Interview-ready SQL, Python, and system design skills --- ## Who This Is For - Analysts or developers who want to move into data engineering - Beginners with no prior database or pipeline experience - Anyone who wants to go from zero to job-ready in one structured program **Prerequisites:** A Windows computer. No prior SQL, Python, or cloud experience required. --- ## Tools Covered | Category | Tools | |----------|-------| | Database | PostgreSQL, pgAdmin | | Query language | SQL (PostgreSQL dialect) | | Programming | Python 3, pandas, requests, psycopg2 | | Web scraping | BeautifulSoup, Playwright | | File formats | CSV, JSON, Excel, Parquet | | Transformation | dbt (data build tool) | | Orchestration | Apache Airflow | | Cloud | Google Cloud (BigQuery, Cloud Storage) or Azure (Blob Storage, Azure SQL) | | Containerisation | Docker, Docker Compose | | Visualisation | Power BI | | Streaming | Apache Kafka | | Version control | Git, GitHub | --- ## 12-Week Curriculum | Week | Theme | Deliverable | |------|-------|-------------| | **1** | Introduction to Data Engineering & SQL Foundations | Assignment: customer database + 20 business questions | | **2** | Intermediate SQL & Database Design | Project: Retail Sales Analytics Database | | **3** | Python for Data Engineering | Project: Sales Data Cleaning Pipeline | | **4** | Data Wrangling & APIs | Project: Weather Data API Pipeline (API → CSV → PostgreSQL) | | **5** | Web Scraping & File Processing | Project: Remote …