ON JOINTLY LEARNING THE PARAMETERS IN A CHARACTER-SYNCHRONOUS INTEGRATED SPEECH AND LANGUAGE MODEL

A joint learning algorithm is proposed in this paper to enhance the in tegrated speech and language model operating in the character-synchron ous mode. A character-synchronous approach is first proposed to integr ate speech and language information, including morphology and parts-of -speech, for ranking the candidates right after each Chinese character is uttered. Since the search space is cut down very efficiently by ap plying high-level knowledge in early time, the character-synchronous s core function enables our system to operate in real time. To further e nhance system performance, a joint learning algorithm is then derived to adjust all parameters of the speech and language processing modules simultaneously, according to their contributions to the overall discr imination power, to minimize the error rate. The proposed approaches a re compared with a baseline system, which directly couples an HMM-base d speech recognizer with a bigram language model. The performance of 7 5.71% character accuracy rate is obtained for the baseline system when it is tested on the task of recognizing 1000 Chinese spoken sentences with a very large vocabulary (90495 words) in the speaker-independent isolated-character mode. The character accuracy rate is improved to 8 8.26% with the character-synchronous approach, and 94.16% after the jo int learning algorithm is applied.