Data Sheet 1_Development and pilot testing of multi-stakeholder questionnaires for assessing University 4.0 transformation in higher education institutions.zip

Introduction

The transition to University 4.0 requires higher education institutions to develop reliable and context-appropriate tools capable of evaluating digital, pedagogical, research, and governance transformations from the perspective of multiple stakeholders. In Morocco, the available evaluation tools remain limited, as they were developed in different institutional contexts or focus on isolated aspects of digital transformation. This study aimed to develop, test, and exploratorily refine a set of multi-stakeholder questionnaires designed to assess the transition to University 4.0 in Moroccan higher education institutions.

Methods

Six questionnaires were developed for students, faculty members and administrative staff, covering Education 4.0, Research 4.0 and Governance 4.0. Their refinement combined pilot tests, psychometric indicators, XAI-based item importance using SHAP and importance by permutation, as well as content validation based on expertise.

Results

Following the combined process of psychometrics, XAI, and expert review, 52 of the initial 183 items were retained, 51 required reformulation, and 80 were recommended for removal. The student questionnaires showed the strongest overall preliminary performance. Student_Education exhibited questionable internal consistency (α = 0.663) and an average corrected item-total correlation of 0.253, while reaching a relatively high S-CVI/Ave of 0.894. Student_Gov_Research demonstrated acceptable internal consistency (α = 0.794), an average corrected item-total correlation of 0.297, close to the conventional criterion of 0.30, and the highest S-CVI/Ave value (0.941).

Discussion

The results support the use of XAI as an exploratory decision-support tool for the refinement of questionnaires when combined with psychometric evidence, theoretical considerations, and expert judgment. The refined instruments remain pilot prototypes and require further validation using larger, balanced, and multi-institutional samples.