Higher-level parallel programming languages can be difficult to implem
ent efficiently on parallel machines. This paper shows how a flexible,
compiler-controlled memory system can help achieve good performance f
or language constructs that previously appeared too costly to be pract
ical. Our compiler-controlled memory system is called Loosely Coherent
Memory (LCM). It is an example of a larger class of Reconcilable Shar
ed Memory (RSM) systems, which generalize the replication and merge po
licies of cache-coherent shared-memory. RSM protocols differ in the ac
tion taken by a processor in response to a request for a location and
the way in which a processor reconciles multiple outstanding copies of
a location. LCM memory becomes temporarily inconsistent to implement
the semantics of C* parallel functions efficiently. RSM provides a co
mpiler with control over memory-system policies, which it can use to i
mplement a language's semantics, improve performance, or detect errors
. We illustrate the first two points with LCM and our compiler for the
data-parallel language C*.