Validating LLM Annotation Without a Gold Standard: A Three-Way Design Applied to Hybrid Arabic–French Corpora — annotation labels, code and materials

Annotation labels, analysis code and coding materials for a three-way validation study of LLM-assisted annotation on Moroccan YouTube comments in Darija, Modern Standard Arabic and French. Two corpora of 600 comments each — the 2030 FIFA World Cup bid and the September 2023 Al Haouz earthquake — were annotated by GPT-4o-mini for sentiment and thematic dimension; a stratified sub-sample of 120 comments was independently coded by two human annotators blind to the model's output. The deposit contains the label-level data sufficient to reproduce every coefficient, confidence interval and figure reported in the accompanying article, together with the collection notebooks, the analysis script and the annotation guide supplied to the coders. Comment texts are not redistributed, in accordance with the platform's API terms and in view of the personal and, for the earthquake corpus, sensitive nature of the material. See README for details.