TatianaSnauwaert/Pump-it-up

Prediction of faulty pumps in Tanzania # Pump-it-up project ### Can you predict which water pumps are faulty? ## Goal Using data from Taarifa and the Tanzanian Ministry of Water, predict which pumps are functional, which need some repairs, and which don't work at all based on a number of variables about what kind of pump is operating, when it was installed, and how it is managed. A smart understanding of which waterpoints will fail can improve maintenance operations and ensure that clean, potable water is available to communities across Tanzania. The logbook of the project can be found here. All notebooks can also be found on Kaggle in my profile. This repo includes the following files: - The source files can be found in the folder named *Input data* (the input data can also be found on Kaggle here); - Pump it up_I. EDA (exploratory data analysis); - Pump it up_II. Data cleaning and preprocessing; - Pump it up_III. Model selection and submission. In my research I've first performed an **exploratory data analysis**. In the beginning I calculated a preliminary/baseline accuracy score which means that a model predicting with the accuracy less than 54.31% is not adding any value, so it would not be better than an uneducated guess. I then splited the data into numerical and categorical columns, identified missing values to deal with in the preprocessing phase, searched for outliers in the data and assessed correlations among attributes. In the next step I've performed **data cleaning and preprocessing**. First of all I dropped features containing similar information to avoid multicollinearity. Then I filled missing values, reduced cardinality of several categorical features that had many types of values to be able to encode them. I performed ordinal encoding for those variables where it made sense and one-hot encoding for the rest of variables. Finally, I used feature engineering to create new predictors (including LDA, binning, binary variables, turning a date-time variable into a continious numerical …