Experimental validation of Specialised Questioning Techniques in Conservation

Abstract

Conservation increasingly relies on social science tools to understand human behaviour. Specialised Questioning Techniques (SQTs) are a suite of methods designed to reduce bias in social surveys, and are widely used to collect data on sensitive topics, including compliance with conservation rules. Most SQTs have been developed in western, industrialised, educated, rich and democratic countries (so called WEIRD contexts), meaning their suitability in other contexts may be limited. Whether these techniques perform better than conventional direct questioning is important for those considering their use. Here, we adopt an experimental design to validate the performance of four SQTs (Unmatched Count Technique, Randomised Response Technique, Crosswise model, Bean method) against direct questions when asking about a commonly researched sensitive behaviour in conservation, wildlife hunting. We developed fictional characters, and for each method, asked respondents to report the answers that each fictional character should give when asked if they hunt wildlife. With data collected from 609 individuals living close to protected areas in two different cultural and socio-economic contexts (Indonesia, Tanzania), we quantified the extent to which respondents understood and followed SQT instructions and explored the socio-demographic factors that influenced whether they provided a correct response. Participants were more likely to refuse SQTs than direct questions and modelling suggested SQTs were harder for participants to understand. Demographic factors, including age and education level significantly influenced response accuracy. When sensitive responses were required, all SQTs (excluding Bean method) outperformed direct questions, demonstrating that SQTs can successfully reduce sensitivity bias. However, when asked about each method, most respondents (59-89%) reported they would feel uncomfortable using them to provide information on their own hunting behaviour, highlighting the considerable challenge of encouraging truthful reporting on sensitive topics. This work demonstrates the importance of assessing the suitability of social science methods prior to their implementation in conservation contexts.

Data

Here, we provide all the data and R code to conduct the analysis included in this paper. There are three separate dataset files, each containing a code-book describing the data. The respondent ID correspond across these files.

demographics & set review.R

This file includes R code for conducting basic descriptive statistics on the demographic data contained in the main-data.xls file. It also includes code summarising respondents reviews of each method, using the set-review.xls file.

glmms.R

This file includes the R code for running the GLMMs for each country and for plotting the data. It uses the main-data.xls dataset.

randomiser-results.R

Code to summarising how often each dice & button were reported as rolled/selected in the RRT. Uses the randomiser-results.xls.

sqt-review.xls

This dataset relates to the part of the questionnaire where respondents where asked to review how easy, how private, how well they understood and how comfortable they would feel using the SQT to provide information about their own behaviour.

main-data.xls

This dataset contains one row for each test (character) of a method. In total, respondent answered 15 questions each (3 characters, for five methods). The sample sizes were N=303 in Indonesia, N=306 in Tanzania.

randomiser-results.xls

This dataset provides information on the outcome of each randomising event (e.g. dice roll, button selected) each respondent was involved in.