A Multitask Pretrained Language Model for Holistic Quality Evaluation of Curriculum and Examinations in Higher Education

Abstract

Objectives: This study aims to identify and validate quality criteria for curriculum and examinations in Tanzanian higher education, and to develop a multitask pretrained language model that holistically assesses multiple quality dimensions simultaneously using enhanced Natural Language Processing (NLP). Method: A mixed-methods design was employed. Qualitative data were collected via semi-structured interviews with quality assurance personnel from 10 of 13 identified Tanzanian universities and the Tanzania Commission for Universities (TCU). Thematic analysis identified quality dimensions and indicators. Computationally, the all-MiniLM-L12-v2 (SBERT) model was fine-tuned in two phases: Phase 1 used 104,000 contrastive sentence pairs from computer science textbooks and open educational resources (OER); Phase 2 trained task-specific heads on 6,636 labelled sentences for Bloom's Taxonomy classification, learning-outcome coherence detection, and industry-skills relevance matching. Findings: The fine-tuned multitask model achieved a test accuracy of 97.87%, an F1 score of 97.93%, and a recall of 99.65%, outperforming prior NLP approaches on comparable educational quality tasks. The SBERT + SpaCy combined approach yielded a contextual similarity score of 0.6895 versus 0.4860 for plain SBERT, confirming enhanced deep contextual comprehension. The model demonstrated robust performance in evaluating three quality dimensions: cognitive-level alignment with Bloom's Taxonomy, coherence between learning outcomes and instructional methods, and relevance of learning outcomes to industry skill standards (ISCO). Novelty: This study introduces a novel multitask PLM framework that simultaneously evaluates interrelated quality dimensions of curriculum and examinations, addressing the fragmented evaluation approaches prevalent in existing literature. The integration of SpaCy-based Named Entity Recognition (NER) with SBERT embeddings, combined with a chapter-level contrastive fine-tuning strategy, yields richer contextual representations deployable in resource-constrained environments.

Keywords: Quality of education, NLP, Pretrained Language Models, Multitask Learning, Curriculum, Examinations, Bloom's Taxonomy, Higher Education