Exploiting the benefits of multiple-path network in DSM systems: Architectural alternatives and performance evaluation

Modern high-performance networks being used for scalable distributed shared -memory (DSM) systems support multiple paths to increase bandwidth and/or r educe contention. Such networks violate the constraint of pairwise in-order message delivery implicitly required by many existing directory-based cach e coherence protocols. To solve this problem, two alternative strategies ar e currently used by computer architects. The first strategy, used in the SG I Origin series, is to employ an intelligent cache coherence protocol which detects and resolves all race conditions caused by out-of-order (OoO) even ts. The second strategy, used in the HAL Mercury series, is to use a sophis ticated network interlace (NI) which detects and remedies every OoO event b efore the messages are fed to the cache coherence controllers. Both strateg ies involve complicated hardware logic, either at the cache coherence contr oller level or at the NI level. In this paper, we propose a new strategy th at uses block correlated FIFO channels. This new strategy detects all poten tial race conditions and prevents them from occurring. It allows the use of a simple cache coherence protocol and an inexpensive NI. We also present a n efficient implementation of this strategy based on current technology. De tailed simulations are performed using benchmark applications to evaluate t he performance of our new strategy. The results indicate that, compared to the existing strategies, our new strategy always provides either the best o r close to the best overall performance. This study also provides valuable insights into the design trade-offs in incorporating modern networks into D SM systems.