A Quality Evaluation Meta-Model for Open Source Software

Abstract

In recent years, Open Source Software (OSS) has gained an increasing attention due to its voluntary supporters, growing community, no vendor lock-in, low total cost of ownership, and ease of accessibility in cloud repositories. In turn, specifying and evaluating OSS quality has become a significant challenge for OSS adoption in organizations that are inclined to use them. Although many OSS quality models (OSS-QMs) have been proposed in literature, the dynamic and diverse nature of OSS has caused these models to be heterogeneous in terms of structure and content. This has adversely affected the standardization of evaluations and led to the evaluation results obtained from different OSS-QMs for the same purpose to be incomparable and sometimes unreliable. Standardization in OSS quality is of vital importance as a communication vehicle for stakeholders in identifying and selecting high-quality products. In this context, meta-modeling can help to define a standardized language and enable to propose quality models with comparable measurements. Therefore, in this thesis, a meta-model for OSS quality (OSS-QMM), which employs a unified structure from existing OSS-QMs and enables the derivation of homogeneous models, has been proposed. For this purpose, a systematic ii and laborious effort has been spent via step-based meta-model creation process, including review-and-revise iterations. In order to validate the OSS-QMM, case study and expert opinion methods have been applied to answer three research questions (RQs) targeted to investigate results comparability, effectiveness, and practical applicability of using the meta-model. Multiple and embedded case study design has been employed for evaluating three real ERP systems, and 20 subject matter experts have been interviewed during the validation process. The results of multi-faceted empirical studies have indicated that the OSS-QMM have addressed solving problems in the OSS quality evaluation and its adoption with high degrees of confidence.