A NEW PARALLEL (PM)-M-3 CODE FOR VERY LARGE-SCALE COSMOLOGICAL SIMULATIONS

We have developed a parallel Particle-Particle, Particle-Mesh ((PM)-M- 3) simulation code for the Cray T3E parallel supercomputer that is wel l suited to studying the time evolution of systems of particles intera cting via gravity and gas forces in cosmological contexts. The paralle l code is based upon the public-domain serial Adaptive (PM)-M-3-SPH (h ttp://coho.astro.uwo.ca/pub/hydra/hydra.html) code of Couchman et al. (1995)[ApJ, 452, 797]. The algorithm resolves gravitational forces int o a long-range component computed by discretizing the mass distributio n and solving Poisson's equation on a grid using an FFT convolution me thod, and a short-range component computed by direct force summation f or sufficiently close particle pairs. The code consists primarily of a particle-particle computation parallelized by domain decomposition ov er blocks of neighbour-cells, a more regular mesh calculation distribu ted in planes along one dimension, and several transformations between the two distributions. The load balancing of the (PM)-M-3 code is sta tic, since this greatly aids the ongoing implementation of parallel ad aptive refinements of the particle and mesh systems. Great care was ta ken throughout to make optimal use of the available memory, so that a version of the current implementation has been used to simulate system s of up to 10(9) particles with a 1024(3) mesh for the long-range forc e computation. These are the largest Cosmological N-body simulations o f which we are aware. We discuss these memory optimizations as well as those motivated by computational performance. Performance results are very encouraging, and, even without refinements, the code has been us ed effectively for simulations in which the particle distribution beco mes highly clustered as well as for other non-uniform systems of astro physical interest. (C) 1998 Elsevier Science B.V. All rights reserved.