Implementation of parallel data mining on an ATM-connected PC cluster and performance analysis of TCP retransmission mechanisms

A recent tendency in parallel computer design has been to use general-purpo se components for system configuration elements such as CPUs, disks, and me mories, which used to be specially developed. Although the connection netwo rk between the processors has been specially developed, it is now possible to configure a large-scale PC cluster with good performance at low cost by making use of an ATM network as a processor connection network because of t he development and cost reduction of ATM network technologies in the commun ication field. In this paper, a large-scale PC cluster is constructed by co nnecting 100 personal computers by means of a general-purpose ATM network. Applications to parallel data mining are evaluated and discussed. In partic ular, an analysis is carried out with a focus on the effect of TCP retransm ission with cell discarding of the ATM switch on the performance. The param eter setting of a retransmission mechanism suitable for the parallel proces sing in the cluster is found. Further, by developing a method for setting t he retransmission spacing parameters to random values for each node, it is shown that a further improvement is possible. (C) 1999 Scripta Technica.