The Impact of Coder Reliability on Reconstructing Archaeological Settlement Patterns from Satellite Imagery: a Case Study from South Africa

Abstract A large archaeological remote sensing project is underway to digitize and classify the pre‐colonial stone walled structures (SWS) on Google Earth satellite imagery in the southern part of Gauteng Province, South Africa. Over 7000 such SWS have been digitized in a study area of some 8000 km 2 . Several research assistants have been involved in classifying the structures. The problem is that different analysts may assign the same SWS to different types and even digitize their outline differently no matter how well they have been trained. Such inter‐analyst variability is a common problem in many fields. In order to minimize its impact, a thorough study of coder reliability in classification of remotely sensed Iron Age SWS has been initiated. The results show unacceptably high variability in the classification of individual SWS. Several contributing factors have been identified. Surprisingly, at the regional level, relatively high levels of inter‐analyst agreement are seen in the same data. The reason probably has to do with strong agreement on the classification of the most diagnostic structures. This may suffice to produce the replicable results at the regional level. Copyright © 2015 John Wiley & Sons, Ltd.