Sign Language Recognition by Combining Statistical DTW and Independent Classification

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 30; no. 11; pp. 2040 - 2046
• Main Authors: Lichtenauer, J.F., Hendriks, E.A., Reinders, M.J.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01-11-2008; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D/stereo scene analysis; Algorithms; Applied sciences; Artificial Intelligence; Classification; Classifier design and evaluation; Classifiers; Computer science; control theory; systems; Computer vision; Computing Methodology; Data Interpretation, Statistical; Degradation; Detectors; Distortion; Exact sciences and technology; Face and gesture recognition; Face recognition; Feature recognition; Handicapped aids; Handwriting recognition; Hidden Markov models; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Information Storage and Retrieval - methods; Mapping; Markov processes; Mathematical models; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Real-time systems; Recognition; Shape; Sign language; Speech and sound recognition and synthesis. Linguistics; Speech recognition; Time series analysis; Vision and Scene Understanding; Warpage; Warping; Time series analysis; Artificial Intelligence; Markov processes; Vision and Scene Understanding; Real-time systems; Computing Methodology; Face and gesture recognition; Classifier design and evaluation; 3D/stereo scene analysis; Markov process; Warping; classifier design and evaluation; statistical dynamic programming; Modeling; Classification; Facies; Hybrid model; Dynamic programming; Stereopsis; Pattern analysis; Time analysis; Multimodel; Statistical analysis; Probabilistic approach; Gesture recognition; Time series; Markov model; Real time system; face and gesture recognition; Sign language; Scene analysis; real-time systems; Hidden Markov model; Manuscript character; Artificial intelligence; Facial expression
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2008.123

To recognize speech, handwriting, or sign language, many hybrid approaches have been proposed that combine dynamic time warping (DTW) or hidden Markov models (HMMs) with discriminative classifiers. However, all methods rely directly on the likelihood models of DTW/HMM. We hypothesize that time warping and classification should be separated because of conflicting likelihood modeling demands. To overcome these restrictions, we propose using statistical DTW (SDTW) only for time warping, while classifying the warped features with a different method. Two novel statistical classifiers are proposed - combined discriminative feature detectors (CDFDs) and quadratic classification on DF Fisher mapping (Q-DFFM) - both using a selection of discriminative features (DFs), and are shown to outperform HMM and SDTW. However, we have found that combining likelihoods of multiple models in a second classification stage degrades performance of the proposed classifiers, while improving performance with HMM and SDTW. A proof-of-concept experiment, combining DFFM mappings of multiple SDTW models with SDTW likelihoods, shows that, also for model-combining, hybrid classification can provide significant improvement over SDTW. Although recognition is mainly based on 3D hand motion features, these results can be expected to generalize to recognition with more detailed measurements such as hand/body pose and facial expression.