This article dears with both exploration and interpretation problems relate
d to posterior distributions for mixture models. The specification of mixtu
re posterior distributions means that the presence of Ic! modes is known im
mediately. Standard Markov chain Monte Carlo (MCMC) techniques usually have
difficulties with well-separated modes such as occur here; the MCMC sample
r stays within a neighborhood of a local mode and fails to visit other equa
lly important modes. We show that exploration of these modes can be imposed
using tempered transitions. However, if the prior distribution does not di
stinguish between the different components, then the posterior mixture dist
ribution is symmetric and standard estimators such as posterior means canno
t be used. We propose alternatives for Bayesian inference for permutation i
nvariant posteriors, including a clustering device and alternative appropri
ate loss functions.