SPIFFI - A SCALABLE PARALLEL FILE SYSTEM FOR THE INTEL PARAGON

This paper presents the design and performance of SPIFFI, a scalable h igh-performance parallel file system intended for use by extremely I/O intensive applications including ''Grand Challenge'' scientific appli cations and multimedia systems. This paper contains experimental resul ts from a SPIFFI prototype on a 64 node/64 disk Intel Paragon. The res ults show that SPIFFI provides high performance and linear scaleup on real hardware. The paper also explains how shared file pointers (i.e., file pointers that are shared by multiple processes) can simplify the design of a parallel application. By sequentializing 110 accesses and by providing dynamic 110 load balancing, a shared file pointer may ev en improve an application's performance. This paper also presents the predictions of a SPIFFI simulator that we validated using the prototyp e. The simulator results show that SPIFFI continues to provide high pe rformance even when it is scaled to configurations with as many as 128 disks or 256 compute nodes.