Mixing by the Rayleigh-Taylor (R-T) instability in incompressible magn
etic fluids is studied with two- and three-dimensional ideal MHD simul
ations. Evolution and amplification of magnetic fields in the Rayleigh
-Taylor instability are explored in the linear and nonlinear regime fo
r various perturbations. In a single-mode perturbation, both tangentia
l and normal magnetic fields to the density interface decrease the gro
wth of instabilities, while a tangential field is more efficient in st
abilizing the flow, as predicted by linear theory. However, the growth
of a multiple-mode perturbation in the nonlinear regime tends to be e
nhanced by a normal magnetic field because the flow is collimated alon
g the field lines. A strong tangential field suppresses the growth of
short-wavelength modes, as expected from linear theory. Three-dimensio
nal numerical simulations are carried out to study the amplification o
f magnetic fields in the turbulent mixing layer developed by the insta
bility in the nonlinear regime. We find the following results. (1) The
turbulent flow amplifies the magnetic held more efficiently in three-
dimensions than in two-dimensions. (2) The peak of the magnetic energy
spectrum occurs at high wavenumbers, which means that the field is am
plified preferentially on small scales. (3) The peak of the kinetic en
ergy spectrum is at scales significantly longer than the grid scale, i
s independent of grid resolution, and reflects the characteristic size
of the R-T fingers at a given time. (4) Secondary Kelvin-Helmholtz in
stabilities generate vortex rings and ring-structured magnetic fields.
However, the stretching of field lines by R-T fingers is the dominant
amplification mechanism. (5) The final structures are very sensitive
to the initial magnetic field and numerical resolution. (6) The magnet
ic field component along the gravity vector dominates as the instabili
ties grow. This dominance is stronger in three dimensions than in two
dimensions, but it becomes less dominant as the resolution increases.
(7) higher resolution produces a greater total magnetic energy and a s
maller total kinetic energy.