This paper proposes an edge-directed interpolation algorithm for natural im
ages. The basic idea is to first estimate local covariance coefficients fro
m a low-resolution image and then use these covariance estimates to adapt t
he interpolation at a higher resolution based on the geometric duality betw
een the low-resolution covariance and the high-resolution covariance. The e
dge-directed property of covariance-based adaptation attributes to its capa
bility of tuning the interpolation coefficients to match an arbitrarily ori
ented step edge. A hybrid approach of switching between bilinear interpolat
ion and covariance-based adaptive interpolation is proposed to reduce the o
verall computational complexity. Two important applications of the new inte
rpolation algorithm are studied: resolution enhancement of grayscale images
and reconstruction of color images from CCD samples. Simulation results de
monstrate that our new interpolation algorithm substantially improves the s
ubjective quality of the interpolated images over conventional linear inter
polation.