EXPERIMENTAL EVALUATION OF A SIMULATION ENVIRONMENT FOR INFORMATION-SYSTEMS DESIGN

This article presents an experiment assessing the decision support val ue of a simulation environment for the information systems (IS) design process. We have implemented a prototype simulation environment that uses data flow diagrams (DFDs) augmented with the performance rates of system components to specify the structure and dynamics of IS designs . The DFD-based representation is automatically mapped to a stochastic queuing network simulation. Knowledge-based help supports formulation of simulation run parameters and interpretation of output. We measure the prototype's impact on system dynamics assessment via the accuracy of IS professionals' responses to questions about the dynamics of fou r IS cases. The prototype's simulation capability has a significant po sitive effect on accuracy scores for questions involving waiting times , system times, and queue lengths of jobs or customers. Subjects choos ing to conduct more and longer simulation runs provide significantly m ore accurate assessments than less-active users of simulation. The fin dings suggest that IS professionals can make use of simulation technol ogy within a compact time frame if the proper supports are provided; o n-line help or other methods should be used to encourage users to cond uct sufficiently long simulation runs; and embedding the prototype's c apabilities in a computer-aided software engineering workbench would r esult in better designed information systems.