The nonlinear coupling of two scalar nonlinear Schrodinger (NLS) fields res
ults in nonfocusing instabilities that exist independently of the well-know
n modulational instability of the focusing NLS equation. The focusing versu
s defocusing behavior of scalar NLS fields is a well-known model for the co
rresponding behavior of pulse transmission in optical fibers in the anomalo
us (focusing) versus normal (defocusing) dispersion regime [19], [20]. For
fibers with birefringence (induced by an asymmetry in the cross section), t
he scalar NLS fields for two orthogonal polarization modes couple nonlinear
ly [26]. Experiments by Rothenberg [32], [33] have demonstrated a new type
of modulational instability in a birefringent normal dispersion fiber, and
he proposes this cross-phase coupling instability as a mechanism for the ge
neration of ultrafast, terahertz optical oscillations. In this paper the no
nfocusing plane wave instability in an integrable coupled nonlinear Schrodi
nger (CNLS) partial differential equation system is contrasted with the foc
using instability from two perspectives: traditional linearized stability a
nalysis and integrable methods based on periodic inverse spectral theory. T
he latter approach is a crucial first step toward a nonlinear, nonlocal und
erstanding of this new optical instability analogous to that developed for
the focusing modulational instability of the sine-Gordon equations by Ercol
ani, Forest, and McLaughlin [13], [14], [15], [17] and the scalar NLS equat
ion by Tracy, Chen, and Lee [36], [37], Forest and Lee [:18], and McLaughli
n, Li, and Overman [23], [24].