The probe-to-sample separation in near-field scanning optical microsco
pes can be regulated by a noncontact shear-force sensing technique. Th
e technique requires the measurement of a minute dither motion applied
to the probe. We have characterized an optical detection method for m
easuring this motion to determine the optimum detection configuration
in terms of sensitivity and stability. A scalar diffraction model of t
he detection method is developed for calculating sensitivity, and expe
rimental results are found to be in good agreement with the theoretica
l predictions. We find that maximum sensitivity and stability cannot b
e achieved simultaneously, and it may be desirable in practice to trad
e sensitivity for enhanced stability. (C) 1995 Optical Society of Amer
ica