The full U-duality symmetry of toroidally compactified M-theory can on
ly be displayed by allowing non-rectangular tori with expectation valu
es of the gauge fields. We construct an E-d(Z) U-duality invariant mas
s formula incorporating non-vanishing gauge backgrounds of the M-theor
y three-form C. We interpret this mass formula from the point of view
of the Matrix gauge theory, and identify the coupling of the three-for
m to the gauge theory as a topological theta term, in agreement with e
arlier conjectures. We give a derivation of this fact from D-brane ana
lysis, and obtain the Matrix gauge theory description of other gauge b
ackgrounds allowed by the Discrete Light-Cone Quantization. We further
show that the conjectured extended U-duality symmetry of Matrix theor
y on Td in the Discrete Light-Cone Quantization has an implementation
as an action of Ed+l(Z) on the BPS spectrum. Some implications for the
proper interpretation of the rank N of the Matrix gauge theory are di
scussed. (C) 1998 Elsevier Science B.V.