We explore the orthogonal decomposition of tensors ( also known as multidim
ensional arrays or n-way arrays) using two different definitions of orthogo
nality. We present numerous examples to illustrate the difficulties in unde
rstanding such decompositions. We conclude with a counterexample to a tenso
r extension of the Eckart-Young SVD approximation theorem by Leibovici and
Sabatier [Linear Algebra Appl., 269 (1998), pp. 307-329].