Area of Applicability for Flood Susceptibility Mapping: Analysis Code

Analysis code accompanying the manuscript: Tella, A., Song, Y., Zahidi, I., Pham, Q. B., & Dewan, A. Toward Reliable GeoAI for Flood Susceptibility Mapping: Integrating Machine Learning with the Area of Applicability in a Rapidly Urbanising City. Scope This deposit implements the Area of Applicability (AoA) framework of Meyer and Pebesma (2021) for flood susceptibility mapping in Kuala Lumpur, Malaysia. Model training and hyperparameter tuning are outside the scope of this deposit. The workflow begins from three already-tuned models supplied as saved .joblib objects (Random Forest, XGBoost and CatBoost), which are loaded in the first block. The purpose of the code is to document the applicability analysis, not the model fitting. The models were developed in an earlier study of the same area (Tella, Zahidi, et al., 2026) and are reused here without refitting, so that the applicability analysis is applied to models with an independently reported performance. Their configuration is reported in Section 3.2 of the manuscript and is also printed by the final block of this notebook. What the code does Starting from the saved models and the predictor stack, the notebook: Computes an importance-weighted dissimilarity index (DI) in the standardised predictor space, separately for each model, using that model's own feature-importance weights. Derives a model-specific AoA threshold from the 95th percentile of the training DI distribution. Maps DI, the applicability mask and relative extrapolation uncertainty across the full study area. Relative uncertainty is DI rescaled by the 99th percentile of its own distribution over the prediction grid and clipped at 1, which is a robust 0-to-1 scaling and is separate from the inside/outside AoA classification. Reports probabilistic evaluation metrics on a held-out test split: LogLoss, Brier score, expected calibration error, PR-AUC, Matthews correlation coefficient and balanced accuracy. Compares ten-fold cross-validation error against spatial prediction error, separately for locations inside and outside the AoA. Assesses dissimilarity as a continuous predictor of prediction error, using a Spearman correlation and a binned trend, alongside a two-sided permutation test on the binary inside/outside split. The notebook is organised into 33 sequentially numbered blocks, each preceded by a short explanation of what it does and why. It is intended to be run top to bottom in order. Required inputs Three things must be in place before the notebook will run: Tuned models — three .joblib files, one each for Random Forest, XGBoost and CatBoost. Predictor rasters — GeoTIFF, all twelve conditioning factors, on a common grid, extent and CRS. Flood inventory — point vector with binary flood / non-flood labels. All file paths are declared together at the top of the notebook and must be edited to point at a local copy of the data before anything else is run. Predictor consistency The dissimilarity index is a distance in predictor space, so it is only meaningful if the prediction rasters carry the same variables, in the same order and the same units, as the data the models were trained on. Confirm before running that the band or column order matches the feature order the models expect, that all rasters share one CRS, resolution, extent and nodata value, and that standardisation uses statistics from the training subset only, never from the full grid. A silent mismatch in feature order will not raise an error. It will produce a plausible-looking but meaningless applicability map. This is the single most likely way to get a wrong result from this notebook. Version compatibility .joblib files are Python pickles, so loading one requires the same libraries that created it. Across major versions of scikit-learn, xgboost or catboost a saved model may fail to load, or load with altered internal state and produce different predictions without an obvious error. Record the library versions used to train the models and load them in a matching environment. Loading a pickle also executes code, so only load .joblib files from a source you trust. Reproducibility Random seeds are set for the train/test split, the cross-validation folds and the permutation test. Results should reproduce exactly on the same data with the same library versions. Small differences across platforms or library versions are possible in the boosting models and in floating-point summaries; the reported correlations and area statistics should be stable to the precision quoted in the manuscript. Outputs Georeferenced GeoTIFF rasters, one set per model: dissimilarity index (continuous), applicability mask (inside / outside AoA), flood susceptibility probability, and relative extrapolation uncertainty. Plus the figures and summary tables reported in the manuscript, a CSV of the DI-versus-error statistics, and a CSV listing the configuration each saved model was fitted with, read from the model objects themselves. One figure in the manuscript is not generated here: the class areas for the low, moderate and high uncertainty categories were computed in ArcGIS Pro from the exported relative-uncertainty rasters, not in the notebook. Data availability The flood inventory (Department of Irrigation and Drainage, Malaysia) and the topographic layers derived from Department of Survey and Mapping Malaysia (JUPEM) data are used under licence and are not redistributed in this deposit. They are available from the respective agencies on reasonable request. Because the input data cannot be shared, the notebook cannot be executed end to end from this deposit alone. It is published so that the applicability methodology is inspectable and transferable to other study areas, not as a one-click reproduction. Applying this to another study area The AoA computation is not specific to Kuala Lumpur or to flooding. To reuse it, replace the inventory, the predictor stack and the saved models, and update the paths block. The threshold is derived from the user's own training distribution, so no constant needs changing. The workflow is also not tied to these three algorithms. Any fitted classifier exposing per-feature importances and a predict_proba method can be substituted, since the dissimilarity index needs only importance weights and the susceptibility maps need only predicted probabilities. Models without a native importance measure, such as a support vector machine or a neural network, would need an alternative weighting scheme, for example permutation importance or SHAP values, in place of the built-in attribute. Requirements Python 3, with numpy, pandas, geopandas, rasterio, scikit-learn, xgboost, catboost, scipy, statsmodels, matplotlib and joblib. xgboost is not imported directly but must be installed for the saved XGBoost model to unpickle. geopandas and rasterio depend on GDAL; if installation via pip gives trouble, install them through conda-forge instead. References Meyer, H., & Pebesma, E. (2021). Predicting into unknown space? Estimating the area of applicability of spatial prediction models. Methods in Ecology and Evolution, 12(9), 1620–1633. doi.org Tella, A., Zahidi, I., Fai, C. M., Okolie, C., & Pham, Q. B. (2026). From prediction to prevention: An explainable GeoAI framework for flood susceptibility and urban exposure assessment using machine and deep learning models. Sustainable Development, 1–20. doi.org