We investigate numerically the appearance of heteroclinic behavior in a thr
ee-dimensional, buoyancy-driven fluid layer with stress-free top and bottom
boundaries, a square horizontal periodicity with a small aspect ratio, and
rotation at low to moderate rates about a vertical axis. The Prandtl numbe
r is 6.8. If the rotation is not too slow, the skewed-varicose instability
leads from stationary rolls to a stationary mixed-mode solution, which in t
urn loses stability to a heteroclinic cycle formed by unstable roll states
and connections between them. The unstable eigenvectors of these roll state
s are also of the skewed-varicose or mixed-mode type and in some parameter
regions skewed-varicose like shearing oscillations as well as square patter
ns are involved in the cycle. Always present weak noise leads to irregular
horizontal translations of the convection pattern and makes the dynamics ch
aotic, which is verified by calculating Lyapunov exponents. In the nonrotat
ing case, the primary rolls lose, depending on the aspect ratio, stability
to traveling waves or a stationary square pattern. We also study the symmet
ries of the solutions at the intermittent fixed points in the heteroclinic
cycle.