WSGatungo/Africa-Energy-Data-Extraction-and-MongoDB-Storage-2000-2024-

Extraction of energy-related data from the Africa Energy Portal for all African countries (2000-2024) and storing it in MongoDB. # Energy Data Extraction and MongoDB Storage (2000–2024) ## Overview This project involves extracting energy-related data from the Africa Energy Portal for all African countries, covering the years 2000 to 2024, and storing it in a MongoDB collection. The data includes various energy indicators such as electricity generation, access, consumption, and renewables. ## Project Structure - `ENERGY DATABASE.ipynb`: Jupyter notebook containing the Python code for data extraction, formatting, and storage. - `Africa Energy Data.csv`: CSV file generated from the formatted data. - `Social Economic.json`, `Electricity.json`, `Energy.json`: JSON files containing raw data from the Africa Energy Portal. ## Instructions ### 1. Data Collection - Navigate through the portal’s dashboards, datasets, and country profiles. - Extract all available energy indicators dataset (e.g., electricity generation, access, consumption, renewables, etc.). - Ensure full coverage for each country and each year from 2000 to 2024. ### 2. Data Formatting - Structure your data using the following columns: - `country`, `country_serial`, `metric`, `unit`, `sector`, `sub_sector`, `sub_sub_sector`, `source_link`, `source`, `2000`, `2001`, ..., `2024` - Each row should represent one metric for one country across all years. - Include the source link and source name for each metric. ### 3. Data Storage - Store the formatted data in a MongoDB collection. - Ensure each document in MongoDB reflects the full schema above. - Use appropriate data types (e.g., strings for text fields, numbers for year values). ### 4. Validation - Check for missing years or metrics and document any gaps. - Ensure consistency in units and naming conventions across countries. ## Setup 1. Install required dependencies: - `pip install pandas pymongo` 2. Ensure MongoDB is installed and running locally (default port 27017). 3. Place the JSON files (`Social Economic.json`, `Electricity.json`, `Energy.json`) in the project director …