Criteria for the evaluation of self-* systems

32nd ACM/IEEE International Conference on Software Engineering - ICSE 2010, Cape Town, South Africa, 1-8 May 2010 International audience In the last years, the growing complexity of the current applications has led to the design of self-adapting systems presenting self-* properties. These systems are composed of several autonomous interactive entities. They behave autonomously and present enhanced characteristics allowing them to handle dynamics coming from exogenous and endogenous changes. In this paper, we propose a set of criteria for the description and evaluation of the adaptive properties of such systems. They aim to provide a concrete mechanism to analyze the quality of the design of adaptive systems, to evaluate the effect of self-* properties on the performances and to compare the adaptive features of different systems. The criteria are grouped into different categories: methodological, architectural, intrinsic, and runtime evaluation. They have been identified and specified by analyzing several case studies, which address self-adaptivity issues through different approaches with different objectives in various application contexts.