For block-based classification, an image is divided into blocks, and a feat
ure vector is formed for each block by grouping statistics extracted from t
he block. Conventional block-based classification algorithms decide the cla
ss of a block by examining only the feature vector of this block and ignori
ng context information, In order to improve classification by context, an a
lgorithm is proposed that models images by two dimensional (2-D) hidden Mar
kov models (HMM's). The HMM considers feature vectors statistically depende
nt through an underlying state process assumed to be a Markov mesh, which h
as transition probabilities conditioned on the states of neighboring blocks
from both horizontal and vertical directions. Thus, the dependency in two
dimensions is reflected simultaneously, The HMM parameters are estimated by
the EM algorithm. To classify an image, the classes with maximum a posteri
ori probability are searched jointly for all the blocks. Applications of th
e HMM algorithm to document and aerial image segmentation show that the alg
orithm outperforms CART(TM), LVQ, and Bayes VQ.