We analyze systematic effects in the determination of the galaxy luminosity
function in clusters using a deep mock catalog constructed from a numerica
l simulation of a hierarchical universe. The results indicate a strong tend
ency to derive a rising faint end (alpha less than or similar to -1.5) in c
lusters selected in two dimensions, using a galaxy catalog constructed with
a universal flat luminosity function with alpha similar or equal to 1.0. T
his is a result of the projection effects inherent in catalogs of clusters
constructed using two-dimensional data. Many of the clusters found in two d
imensions have no significant three-dimensional counterparts, and most suff
er from massive background contamination that cannot be corrected for by su
btracting random offset fields. The luminosity function of high surface bri
ghtness galaxies in the field and within small groups follows a Schechter f
unction with a fairly flat faint-end slope, n(L) proportional to L-alpha, w
ith alpha = -0.9 to -1.2. On the contrary, observational studies of cluster
s constructed using Abell, EDCC, and APM catalogs are systematically found
to have steeper luminosity functions with alpha = -1.4 to -2.0. This may be
attributed to projection effects rather than a dominant population of high
surface brightness dwarf galaxies (M greater than or similar to M* + 2) in
clusters. It should be straightforward to confirm our results by measuring
redshifts of these faint cluster galaxies.