Interpretable Measures for Arabic Readability

Readability assessment is a crucial task in natural language processing (NLP), with wide reaching implications in education, accessibility, and language learning. While prior research has primarily focused on predicting readability levels, this work shifts the focus toward understanding why a text is assigned a particular level. By identifying key linguistic features that drive readability, the goal is to enhance interpretability in Arabic text complexity assessment. Arabic poses unique challenges for automatic readability evaluation due to its morphological richness, variation across dialects, and limited availability of annotated corpora. The complex structure of Arabic words, and extensive derivational morphology add layers of difficulty in determining textual complexity. These challenges necessitate a more comprehensive approach to readability evaluation beyond traditional classification. To address these limitations, this work leverages the Balanced Arabic Readability Evaluation Corpus (BAREC)—a large-scale, manually annotated dataset of 68,182 sentences across 19 readability levels, from kindergarten to postgraduate comprehension. BAREC is designed to ensure genre diversity, topical variety, and balanced target audiences, making it a strong foundation for Arabic readability modeling. This study systematically engineers and evaluates a comprehensive set of linguistic features from BAREC’s annotation guidelines, including morphology, syntax, vocabulary, syllables, and content and vocabulary-based measures. Through extensive experimentation, it investigates the effectiveness of these features in explaining and predicting readability levels across three task configurations: foundational levels (1–11), advanced levels (12–19), and the full 19-level scale. In this study, we explore a variety of modeling approaches, including traditional machine learning classifiers, a fully interpretable rule-based model, and an AraBERT model based on transformers. Furthermore, a hybrid model is proposed that integrates linguistic features with contextual embeddings to improve both performance and transparency. The results demonstrate that linguistic features significantly enhance interpretability, especially in low-resource settings, and that feature-driven models can provide human readable justifications aligned with expert annotation practices. This research not only advances Arabic readability prediction but also contributes interpretable insights into text complexity, paving the way for more accessible and pedagogically meaningful NLP tools in Arabic education.