Performance-oriented code development, software testing, performance analysis and parameter studies for distributed and parallel systems commonly require to conduct a large number of executions. Every execution of an application can be viewed as a scientific experiment. So far there exists very little support to specify and to control execution of a large number of experiments. Various problems must be addressed, such as which input files to read, where to store program's output, what performance metrics to measure and what range of problem parameters to observe. This paper describes ZEN, a directive-based language to support automatic experiment management for a wide variety of parallel and distributed architectures. It is used to specify arbitrarily complex program executions in the context of performance analysis and tuning, parameter studies, and software testing. ZEN introduces directives to substitute strings and insert assignment statements inside arbitrary files, such as program, input, script, or makefiles. This enables the programmer to invoke experiments for arbitrary value ranges of any problem parameter, including program variables, file names, compiler options, target machines, machine sizes, scheduling strategies, data distributions, etc. The number of experiments can be controlled through ZEN constraint directives. Finally, the programmer may request a large set of performance metrics to be computed for any code region of interest. The scope of ZEN directives can be restricted to arbitrary file or code regions. We have implemented a prototype tool that supports the user by employing ZEN directives to control and manage parameter studies and performance experiments for distributed and parallel programs on cluster architectures. We report results of using our prototype implementation for performance analysis of an ocean simulation application and for parameter study of a computational finance code.

Top