Satellite-Based Human Settlement Datasets Inadequately Detect Refugee Settlements: A Critical Assessment at Thirty Refugee Settlements in Uganda

Satellite-based broad-scale (i.e., global and continental) human settlement data are essential for diverse applications spanning climate hazard mitigation, sustainable development monitoring, spatial epidemiology and demographic modeling. Many human settlement products report exceptional detection accuracies above 85%, but there is a substantial blind spot in that product validation typically focuses on large urban areas and excludes rural, small-scale settlements that are home to 3.4 billion people around the world. In this study, we make use of a data-rich sample of 30 refugee settlements in Uganda to assess the small-scale settlement detection by four human settlement products, namely, Geo-Referenced Infrastructure and Demographic Data for Development settlement extent data (GRID3-SE), Global Human Settlements Built-Up Sentinel-2 (GHS-BUILT-S2), High Resolution Settlement Layer (HRSL) and World Settlement Footprint (WSF). We measured each product’s areal coverage within refugee settlement boundaries, assessed detection of 317,416 building footprints and examined spatial agreement among products. For settlements established before 2016, products had low median probability of detection and F1-score of 0.26 and 0.24, respectively, a high median false alarm rate of 0.59 and tended to only agree in regions with the highest building density. Individually, GRID3-SE offered more than five-fold the coverage of other products, GHS-BUILT-S2 underestimated the building footprint area by a median 50% and HRSL slightly underestimated the footprint area by a median 7%, while WSF entirely overlooked 8 of the 30 study refugee settlements. The variable rates of coverage and detection partly result from GRID3-SE and HRSL being based on much higher resolution imagery, compared to GHS-BUILT-S2 and WSF. Earlier established settlements were generally better detected than recently established settlements, showing that the timing of satellite image acquisition with respect to refugee settlement establishment also influenced detection results. Nonetheless, settlements established in the 1960s and 1980s were inconsistently detected by settlement products. These findings show that human settlement products have far to go in capturing small-scale refugee settlements and would benefit from incorporating refugee settlements in training and validating human settlement detection approaches.