Margin-maximizing classification of sequential data with infinitely-long temporal dependencies

• Published in: Expert systems with applications Vol. 40; no. 11; pp. 4519 - 4527
• Main Author: Chatzis, Sotirios P.
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Ltd 01-09-2013; Elsevier
• Subjects: Algorithms; Applied sciences; Computational efficiency; Computer science; control theory; systems; Data processing. List processing. Character string processing; Decision theory. Utility theory; Exact sciences and technology; Information systems. Data bases; Margin maximization; Mathematical analysis; Mathematical models; Mathematical programming; Maximization; Mean-field principle; Memory organisation. Data processing; Military technology; Operational research and scientific management; Operational research. Management science; Sequence memoizer; Sequential data modeling; Software; Temporal logic; Training; Mean-field principle; Sequential data modeling; Margin maximization; Sequence memoizer; Costs; Multiobjective programming; Temporal databases; Non parametric estimation; Electronic countermeasure; Modeling; Convex programming; Markov chain; Mean-field theory; Efficiency; Vector support machine; Convex function; Learning algorithm; Generative model; Bayes estimation; Local search; Mean field approximation; Data dependency; Dependability; Pattern recognition; Trapping; Data models; Objective function; Convex analysis
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2013.01.051

► We present a method for sequential data modeling. ► Our approach models temporal dependencies of infinite length. ► It employs a margin maximization training scheme. ► We evaluate it in computer vision applications.
Generative models for sequential data are usually based on the assumption of temporal dependencies described by a first-order Markov chain. To ameliorate this shallow modeling assumption, several authors have proposed models with higher-order dependencies. However, the practical applicability of these approaches is hindered by their prohibitive computational costs in most cases. In addition, most existing approaches give rise to model training algorithms with objective functions that entail multiple spurious local optima, thus requiring application of tedious countermeasures to avoid getting trapped to bad model estimates. In this paper, we devise a novel margin-maximizing model with convex objective function that allows for capturing infinitely-long temporal dependencies in sequential datasets. This is effected by utilizing a recently proposed nonparametric Bayesian model of label sequences with infinitely-long temporal dependencies, namely the sequence memoizer, and training our model using margin maximization and a versatile mean-field-like approximation to allow for increased computational efficiency. As we experimentally demonstrate, the devised margin-maximizing construction of our model, which leads to a convex optimization scheme, without any spurious local optima, combined with the capacity of our model to capture long and complex temporal dependencies, allow for obtaining exceptional pattern recognition performance in several applications.