In this paper, we explore the feasibility of implementing surface continuit
y and ordering constraints by tracking connected local peak points in the s
pace of matching similarity. We present a new image matching algorithm base
d on match paths directly extracted in the similarity space. The main advan
tage of these kinds of tokens over tokens extracted from the left and right
images is that the geometric distortions ca used by perspective effects an
d description inconsistency caused by independent extraction of tokens in t
he left and right images are automatically eliminated. Through the tracking
process, the computation is greatly reduced. In addition, global informati
on available to support a local match for resolving matching ambiguities is
fully utilized in such a way that unrelated global information is excluded
. Thus, the new image matching algorithm is reliable and efficient. By elim
inating the interpolation process at the levels except the finest, the occl
uded regions and depth discontinuities can be well localized.