Early vision algorithms often have a first stage of linear-filtering t
hat 'extracts' from the image information at multiple scales of resolu
tion and multiple orientations. A common difficulty in the design and
implementation of such schemes is that one feels compelled to discreti
ze coarsely the space of scales and orientations in order to reduce co
mputation and storage costs. This discretization produces anisotropies
due to a loss of translation-, rotation-, and scaling-invariance that
makes early vision algorithms less precise and more difficult to desi
gn. This need not be so: one can compute and store efficiently the res
ponse of families of linear filters defined on a continuum of orientat
ions and scales. A technique is presented that allows 1) computing the
best approximation of a given family using linear combinations of a s
mall number of 'basis' functions; 2) describing all finite-dimensional
families, i.e., the families of filters for which a finite dimensiona
l representation is possible with no error. The technique is based on
singular value decomposition and may be applied to generating filters
in arbitrary dimensions and subject to arbitrary deformations; the rel
evant functional analysis results are reviewed and precise conditions
for the decomposition to be feasible are stated. Experimental results
are presented that demonstrate the applicability of the technique to g
enerating multi-orientation multi-scale 2D edge-detection kernels. The
implementation issues are also discussed.