TahaMerzouki/water-dep

Motivated by the persistent water crisis in Morocco, I initiated this project to leverage SDG6 (Sustainable Development Goal 6) data to inform decision-making in water resource management. # Morocco National Water Consumption Analysis ## Project Overview This project analyzes national water consumption data for Morocco using SDG6 (Sustainable Development Goal 6: Clean Water and Sanitation) data. It leverages Azure cloud services for data engineering, processing, and analysis, as well as Power BI for data visualization and insights. ## Data Source Data is sourced from the SDG 6 Data Portal (sdg6data.org). Usage of this data is subject to specific terms and conditions. Please see DATA_USAGE.md for detailed information on data usage rights and citation requirements. ## Project Architecture This project follows an architecture very similar to the one shown in the image below: > [!NOTE] > I'll make sure to provide a more accurate version soon. > This is just the baseline of the project, and I plan to expand it further, so the architecture will evolve over time. ## Azure Services Used - Azure Data Lake Storage Gen2 (ADLS Gen2): ADLS Gen2 is a highly scalable and secure data lake solution for big data analytics. It provides a hierarchical namespace that makes it easier to organize and manage data, and it supports Azure Blob storage features for cost-effective storage. - Azure Databricks: Azure Databricks is a data analytics platform optimized for the Microsoft Azure cloud services platform. It provides a collaborative environment for big data processing and machine learning, using Apache Spark. - Azure Data Factory: Azure Data Factory is a cloud-based data integration service that allows you to create data-driven workflows for orchestrating and automating data movement and data transformation. It's used in this project to manage and schedule data pipelines. - Azure Key Vault: Azure Key Vault is a cloud service for securely storing and accessing secrets, such as API keys, passwords, certificates, and cryptographic keys. It's used in this project to manage sensitive information securely. - Power BI: Power BI is a business analytics tool by M …