rileym/AfricaSoil

Some of my work on Kaggle's Africa Soil Property Prediction Challenge. # Soil Property Prediction This repository archives of my work for the Kaggle's Africa Soil Property Prediction Contest. The files are in more or less the same state as they were the day the competition ended. I placed tenth. *You can view any IPython notebook (.ipynb) through nbviewer.* ####Overview of Competition The goal of the competition was to predict five (real-valued) characteristics of a soil sample from that sample's IR spectroscopy measurements and a small number other features. It is simplest to think of the competition as five separate (or at least I chose too treat each response separately) regression problems. ####The Predictor Variables * The features are split between spectral data (the large majority of the features) and background data, which record the soil’s location, depth, etc. * The spectral data is high dimensional * The spectral data is functional data, and therefore there's lots of autocorrelation ####What I Tried #####Preprocessing * Standardization of the input * Correlation filtering of the input * A home-brew filter/dim-reduction scheme that tried to find representative (by local auto-correlation) frequencies * Transformations of the response * Log * Inverse hyperbolic sine #####Models/Function Classes * Linear: * Lasso * Ridge * Elastic-Net * Supervised Principle Components * Fussed Lasso -- I had high hopes for this one! :'-( * Non-linear: * SVR (with various kernels) * GBM * kNN (only after dim-reduction) #####Post-processing * Before scoring * Inverse transformations of the response (if transformed originally) * Truncation at max or min values * After scoring * Model averaging (I did not get a change to could build a proper meta-model) ####What Worked & General Observations As one would expect, different combinations of approaches worked to different effect on the different response variables. In the end ridge regression and support vector regression performed best. * Ridge regression was relatively successful in …