Acoustic Modelling Using Continuous Rational Kernels

• Published in: 2005 IEEE Workshop on Machine Learning for Signal Processing pp. 165 - 170
• Main Authors: Layton, M.L., Gales, M.J.F.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2005
• Subjects: Biological system modeling; Dynamic programming; Heuristic algorithms; Hidden Markov models; Kernel; Maximum likelihood estimation; Sequences; Speech recognition; Support vector machines; Transducers
• ISBN: 9780780395176; 0780395174
• ISSN: 1551-2541; 2378-928X
• DOI: 10.1109/MLSP.2005.1532893

There has been significant interest in developing alternatives to hidden Markov models (HMMs) for speech recognition. In particular, interest has been focused upon models that allow additional dependencies to be incorporated. One such model is the augmented statistical model. Here a local exponential approximation, based upon derivatives of a base distribution, is made about some distribution of the base model. Augmented statistical models can be trained using a maximum margin criterion, which may be implemented using an SVM with a generative kernel. Calculating derivatives of the base distribution, in particular higher-order derivatives, to form the generative kernel requires complex dynamic programming algorithms. In this paper a new form of rational kernel, a continuous rational kernel is proposed. This allows elements of the generative kernel, including those based on higher-order derivatives, to be computed using standard forms of transducer within a rational kernel framework. In addition, the derivatives are shown to be a principled method of defining marginalised kernels. Continuous rational kernels are evaluated using a large vocabulary continuous speech recognition (LVCSR) task