Context: Testing is a key concern when developing process-oriented solutions as it supports modeling experts who have to deal with increasingly complex models and scenarios such as cross-organizational processes. However, the complexity of the research landscape and the diverse set of approaches and goals impedes the analysis and advancement of research and the identification of promising research areas, challenges, and research directions. Hence, a comprehensive and systematic review that covers state of the art research in the process model testing domain is required. Objective: The objective is to analyze process model testing approaches to identify and present, connected research domains and the used terminology. Additionally, research challenges, gaps, and future research directions are identified and discussed. Method: A systematic literature review is conducted to identify interesting areas for future research and to provide an overview of existing work. Over 6300 potentially matching publications were determined during the search (literature databases, selected conferences/journals, and snowballing). Finally, 153 publications from 2002 to 2013 were selected, analyzed, and classified. Results: It was found that the software engineering domain has influenced process model testing approaches (e.g., regarding terminology and concepts), but recent publications are presenting independent approaches. Additionally, historical data sources are not exploited to their full potential and current testing related publications frequently contain evaluations of relatively weak quality. Overall, the publication landscape is unevenly distributed so that over 31 publications concentrate on test-case generation but only 4 publications conduct performance test. Hence, the full potential of such insufficiently covered testing areas is not exploited. Conclusion: This systematic review provides a comprehensive overview of the interdisciplinary topic of process model testing. Several open research questions are identified, for example, how to apply testing to cross-organizational or legacy processes and how to adequately include users into the testing methods.

Top