Alignment properties in ordinal comparison of discrete event dynamic systems

In the design and optimization of discrete event dynamic systems, it is oft en necessary to order alternative designs based on their relative performan ce, i.e., to rank them from best to worst. In this paper, alignment of obse rved performance orders with true orders is considered and properties of th e alignment are investigated. Spearman's rank correlation coefficient is a measure of agreement between the observed performance orders and the true o nes. It is shown that Spearman's coefficient converges exponentially in the simulation time or observation time, which gives a strong evidence of the efficiency of order comparison for discrete event dynamic systems. In the c ontext of simulation, the effect of simulation dependence on the alignment is also discussed. It is found that neither independent simulation nor the scheme of common random numbers (CRN), a popular scheme for variance reduct ion, can yield dominant performance. Finally, numerical examples based on a networking optimization problem are provided to illustrate the convergence of Spearman's coefficient. In these examples, the standard clock (SC) simu lation technique provides much faster convergence than either independent s imulations or CRN simulations. Both the SC and CRN methods use the same ran dom number sequence to drive many events in parallel; however, under SC the events driving the parallel experiments are all identical, whereas under C RN they may be different.