Customizing the protocols used to manage accesses to different data st
ructures within an application can improve the performance of shared-m
emory programs substantially [10, 21]. Existing systems for using cust
omizable protocols are, however, hard to use because they force progra
mmers to rely on low-level mechanisms for manipulating these protocols
. Since several shared-memory systems implement protocols in software
and can support customizable protocols, the development of intuitive a
bstractions for the use of application-specific protocols is important
. We have designed a language, Ace, that integrates support for custom
izable protocols with minimal extensions to C. Ace applications are de
veloped using the standard shared-memory model with a default sequenti
ally consistent protocol. Performance can then be optimized, with mino
r modifications to the application, by experimenting with different pr
otocol libraries. In this paper, we isolate the issues involved in pro
viding language support for using customizable protocols and present n
ovel language abstractions for their easy use. We describe the impleme
ntation of a compiler and runtime system for Ace, and discuss the issu
es involved in their design. We also present measurements that demonst
rate that Ace has good performance compared to an efficient software d
istributed shared-memory system.