This paper first examines the issues related to scheduling loop application
s on a software distributed shared memory (DSM) system. Then, a dynamic sch
eduling scheme is developed based on the examined issues to enhance the per
formance of loop applications on DSM. Compared with previous works, the pro
posed scheme has several specialties. The first is that the workload of pro
cessors can be effectively balanced even when the computational capabilitie
s of processors and the computational needs of threads are not identical. T
he second is it divides thread mapping into two phases, each with one consi
deration, i.e,, load balance or communication cost, and adopts thread migra
tion and exchange in the two phases, respectively. The third is the exploit
ation of data sharing among threads to reduce data-consistency communicatio
n, and the last is to attack the negative effect of the unnecessary inter-n
ode sharing caused by thread re-mapping. The proposed scheme has been imple
mented on a page-based DSM system called Cohesion. Our experiments show tha
t the proposed scheme is more effective to improve the performance of the t
est programs than related schemes.