African-Digital-Academy/Masters-of-Science-in-Data-Engineering

Masters of Science in Data Engineering, Open University, African Digital University # Masters of Science in Data Engineering Masters of Science in Data Engineering African Digital Academy, Open Campus We are developing a 365-day data engineering challenge course that covers various programming languages, data engineering concepts, data modeling, database concepts, data warehousing, data marts, and cloud computing. Here's a suggested outline for this challenge: # Week 1: Introduction to Data Engineering Day 1: Overview of Data Engineering Concepts and Tools Day 2: Setting up Development Environment for Python, R, Scala, and SQL Day 3: Introduction to Data Modeling and Relational Databases Day 4: Building a Simple ETL Pipeline using Python Day 5: Introduction to Cloud Computing for Data Engineering Day 6: Exploring Cloud-based Data Warehousing Solutions Day 7: Hands-on Exercise: Building a Data Pipeline using Python and Cloud # Week 2: Python for Data Engineering Day 8: Data Manipulation and Transformation with Python (Pandas) Day 9: Data Processing with Python (PySpark) Day 10: Building Data Pipelines with Python (Apache Airflow) Day 11: Data Streaming with Python (Apache Kafka) Day 12: Data Quality and Validation in Python Day 13: Real-time Data Processing with Python (Apache Flink) Day 14: Hands-on Exercise: Building a Python-based Data Pipeline using Airflow and Kafka # Week 3: R for Data Engineering Day 15: Introduction to R for Data Engineering Day 16: Data Manipulation and Transformation with R (dplyr) Day 17: Data Processing with R (SparkR) Day 18: Building Data Pipelines with R (drake) Day 19: Data Streaming with R (StreamR) Day 20: Real-time Data Processing with R (data.table) Day 21: Hands-on Exercise: Building an R-based Data Pipeline using drake and SparkR # Week 4: Scala for Data Engineering Day 22: Introduction to Scala for Data Engineering Day 23: Data Manipulation and Transformation with Scala (Apache Spark) Day 24: Building Data Pipelines with Scala (Apache Beam) Day 25: Data Streaming with Scala (Apache Kafka) Day 26: Real-ti …