The stability of low-lying phonon modes in the layered perovskite supe
rconductor Sr2RuO4 has been investigated by inelastic neutron scatteri
ng. The Sigma(3) branch, which ends at the (0.5 0.5 0) zone boundary i
n the mode corresponding to the RuO6 octahedron rotation around c, exh
ibits a sharp drop near the zone boundary. This clearly indicates that
Sr2RuO4 is close to a rotational instability. Although the frequency
of the rotational mode depends only little on temperature, it anomalou
sly broadens with increasing temperature. A flat dispersion of the rot
ation branch along c reveals pronounced two-dimensional character; the
re is almost no coupling between rotational deformations of neighborin
g layers. In contrast, the tilt mode in La2CuO4 that drives the tetrag
onal-to-orthorhombic phase transition does not exhibit any anomalous b
ehavior in Sr2RuO4.