Recursive “concave–convex” Fisher Linear Discriminant with applications to face, handwritten digit and terrain recognition

• Published in: Pattern recognition Vol. 45; no. 1; pp. 54 - 65
• Main Authors: Ye, Q.L., Zhao, C.X., Zhang, H.F., Chen, X.B.
• Format: Journal Article
• Language: English
• Published: Kidlington Elsevier Ltd 2012; Elsevier
• Subjects: Applied sciences; Exact sciences and technology; Extracting features; Fisher Linear Discriminant; Information, signal and communications theory; Recursive “concave–convex” dimension reduction; Signal and communications theory; Signal representation. Spectral analysis; Signal, noise; Telecommunications and information theory; Recursive “concave–convex” dimension reduction; Extracting features; Fisher Linear Discriminant; Performance evaluation; Automatic classification; Image databank; Eigenvalue; Iterative method; Support vector machine; Algorithm; Signal classification; Convex programming; Optimization; Recursive concave-convex dimension reduction; Dimension reduction; Upper bound; Database; Recursive method; Manuscript character
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2011.07.008

In classification, previous studies have shown that an eigenvalue based technique can be cast as an related SVM-type problem and that by solving this SVM-type problem, the performance can be improved significantly. In this paper, we develop a recursive “concave–convex” Fisher Linear Discriminant (DR) (RPFLD) for dimension reduction technique of high-dimensional data to extract as many meaningful features as possible, which incorporates the fundamental idea behind Fisher Linear Discriminant and casts the Fisher Linear Discriminant as a “concave–convex” programming problem based on the hinge loss. The solution of our method follows from solving the related SVM-type optimization problems iteratively, which means the proposed method, can be viewed as the combination of multiple related SVM-type problems. The special formulation of our method provides convenience for constructing sparse multi-class Fisher Linear Discriminant directly. Due to use of a recursive procedure, the number of features available from RPFLD is independent of the number of classes, meaning that in contrast to the original Fisher Linear Discriminant the number of features available from our method has no upper bound. We evaluate our algorithm on the Yale, and ORL face image databases, handwritten digit database and Terrain image dataset. Experimental results show that RPFLD outperforms other Fisher Linear Discriminant algorithms.
► RPFLD casts the FLD as the related SVM-type problems. ► Based on RPFLD, it is easy to construct the sparse multi-class FLD. ► In contrast to FLD, RPFLD has no the limitation on the number of features. ► A recursive procedure is applied to yield multiple projection axes. ► Experiments disclose RPFLD has superior performance to other classic DR algorithms.