On research execution and replication: The case of automated feedback across different countries

Contains the main data file, Rmd script, and html output of the Rmd script.

Article abstract:

We report on an experiment designed to investigate the effects of automated feedback on learning in undergraduate mathematics. The original experiment was carried out in England and New Zealand under laboratory conditions, and we also conducted a conceptual replication of the experiment in an authentic didactical setting in Kenya. The research focus was to compare the impact of three feedback conditions (none, accuracy, accuracy+elaborated) on student learning. Specifically, we were interested in how feedback impacts on a robust misconception that a definite integral measures a geometrical area captured between a function and the x-axis. Our main finding was that feedback did not significantly impact post-test scores, and this finding was replicated. We found that the overall patterns of results across four parts of the experimental design (pre-test, priming items, intervention items, post-test) were replicated. However, there were differences in the details of the post-test scores across the two experiments, and in the open-text comments left by some participants. We consider the implications for our understanding of misconceptions and feedback design, and discuss the added value of replication in our research.