We consider a XY spin system in a hexagonal crystal lattice with latti
ce constants a and c and with competing interactions J(1) and - J(2) a
long the z direction. The model is in the same universality class as t
he ANNNXY one that exhibits an helicoidal structure characterized by a
period q(c). Using renormalization-group analysis carried out to firs
t order in epsilon = d - 4.5, we show that q(c) proportional to (c/a -
(c/a*))(beta q) with an exponent beta(q) characteristic of the Lifsh
itz point vicinitzy. Applications to Tb are shown to be in good agreem
ent with the experiments.