Psychometric Properties of the Creative Participation Assessment administered by occupational therapy undergraduate students in their work integrated learning

INTRODUCTION: Undergraduate occupational therapy students in South Africa use the Creative Participation Assessment during their work-integrated learning to assess clients' level of creative ability. However, the psychometric properties of this assessment tool when administered by undergraduate students have not yet been investigated AIM AND OBJECTIVES: To investigate the inter-rater reliability, intra-rater reliability, internal consistency and construct validity of the Creative Participation Assessment when administered by undergraduate students METHODS: The study used samples collected from third and fourth-year students at a South African tertiary institution. In Study A, prospective data was collected from students based on a case study video in 2023 to assess inter-rater reliability. In Study B, retrospective data was collected from students during their work integrated learning blocks in 2021, 2022 and 2023 to assess intra-rater reliability, internal consistency, and construct validity. Analysis of inter-rater and intra-rater reliability was conducted using inter-class correlation coefficients. Internal consistency was analysed using Cronbach's alpha. An analysis of construct validity was performed using the Rasch model RESULTS: Poor to moderate inter-rater reliability (0.422) was found with improvements when considering only the levels of creative ability (0.575). Excellent intra-rater reliability was indicated, with improved reliability for the fourth year (0.960) compared to the third-year group (0.889). Good construct validity was demonstrated, with a good overall fit to the Rasch model (χ2 probability = 0.187), ordered thresholds across all items, and good item-fit, although some evidence of uni-dimensionality was noted CONCLUSION: These findings support the use of the Creative Participation Assessment as a valid and clinically meaningful assessment within undergraduate occupational therapy training, particularly in mental health settings. They emphasise the need for structured teaching and supervised practice to enhance scoring consistency, while the strong construct validity underscores its value in informing intervention planning and monitoring client progress IMPLICATIONS FOR PRACTICE: • Undergraduate education of creative ability has value informing intervention planning and monitoring client progress. • The Creative Participation Assessment when used during undergraduate training is a valid and meaningful assessment tool. • Further research for occupational therapy student administration of assessments in practice is needed as is structured teaching and supervised practice.