We introduce a new approach for on-line recognition of handwritten wor
ds written in unconstrained mixed style. The preprocessor performs a w
ord-level normalization by fitting a model of the word structure using
the EM algorithm. Words are then coded into low resolution ''annotate
d images'' where each pixel contains information about trajectory dire
ction and curvature. The recognizer is a convolution network that can
be spatially replicated. From the network output, a hidden Markov mode
l produces word scores. The entire system is globally trained to minim
ize word-level errors.