OPTIMISTIC ACTIVE MESSAGES - A MECHANISM FOR SCHEDULING COMMUNICATIONWITH COMPUTATION

Low-overhead message passing is critical to the performance of many ap plications. Active Messages [27] reduce the software overhead for mess age handling: messages are run as handlers instead of as threads, whic h avoids the overhead of thread management and the unnecessary data co pying of other communication models. Scheduling the execution of Activ e Messages is typically done by disabling and enabling interrupts, or by polling the network. This primitive scheduling control, combined wi th the fact that handlers are not schedulable entities, puts severe re strictions on the code that can be run in a message handler. This pape r describes a new software mechanism, Optimistic Active Messages (OAM) , that eliminates these restrictions; OAMs allow arbitrary user code t o execute in handlers, and also allow handlers to block. Despite this gain in expressiveness,OAMs perform as well as Active Messages. We use d OAM as the base for an RPC system, Optimistic RPC (ORPC), for the Th inking Machines CM-5 multiprocessor; it consists of an optimized threa d package and a stub compiler that hides communication details from th e programmer. ORPC is 1.5 to 5 times faster than traditional RPC (TRPC ) for small messages and performs as well as Active Messages (AM). App lications that primarily communicate using large data transfers or are fairly coarse-grained perform equally well, independent of whether AM s, ORPCs, or TRPCs are used. For applications that send many short mes sages, however, the ORPC and AM implementations are up to three times faster than the TRPC implementations. Using ORPC, programmers obtain t he benefits of well-proven programming abstractions such as threads, m utexes, and condition variables, do not have to be concerned with comm unication details, and yet obtain nearly the performance of hand-coded Active Message programs.