Lodewic/vantage-project

Case project for Vantage AI using the public Tanzania waterpump dataset vantage-project ============================== A data science case for Vantage AI Project based on the cookiecutter data science project template . #cookiecutterdatascience Note: This data science project is unfinished so the predictive modelling and classification steps are left out. These scripts' results were not satisfying nor documented. Presented here are the steps that WERE made. This includes, - Conda environment setup - Use of Makefile to run parts of the analysis - Creation of report website using R * Excluding modelling results as these were unfinished * Modelling scripts would take too much time to streamline ## Getting started: One should be up and running as follows: make create_environment source activate vantage-project make requirements Note: you may have to create the `data/` repositories first, you need - data/raw - data/interim - data/external - data/processed To then create processed data and run all rules in the Makefile, activate your environment and then call make all This will, - check your conda environment - get the data from S3 server - clean the data and encode the data for model training - Create a report website under `reports/Report_site/index.html' with some exploratory analysis * Really this was more of a proof of concept. Since it is unfinished it is not included in the Makefile, but... - With your activated environment you can run `python vantage/models/train_model.py * This will train an extremely basic random forest on 70% of the data * Output is not written anywhere * Output presentation in Rmarkdown was removed. To run the computational data transformation steps: make finaldataset This will only create a cleaned up subset of the data and an encoded version of the data to use in Python for classification modelling using one-hot encoding of included categorical data. ### Report To create a small report 'website', run: make report The output can be found under `reports/Report_site/index.html! This is cre …