We present a novel approach to photometric redshifts, one that merges the a
dvantages of both the template-fitting and empirical-fitting algorithms wit
hout any of their disadvantages. This technique derives a set of templates
describing the spectral energy distributions of galaxies from a catalog wit
h both multicolor photometry and spectroscopic redshifts. The algorithm is
essentially using the shapes of the templates as the fitting parameters. Fr
om simulated multicolor data we show that for a small training set of galax
ies we can reconstruct robustly the underlying spectral energy distribution
s even in the presence of substantial errors in the photometric observation
s. We apply these techniques to the multicolor and spectroscopic observatio
ns of the Hubble Deep Field, building a set of template spectra that reprod
uced the observed galaxy colors to better than 10%. Finally, we demonstrate
that these improved spectral energy distributions lead to a photometric re
dshift relation for the Hubble Deep Field that is more accurate than standa
rd template-based approaches.