Analysis and convergence of weighted dimensionality reduction methods

• Published in: Revista Facultad de Ingeniería no. 56
• Main Authors: Juan Carlos Riaño Rojas, Flavio Augusto Prieto Ortiz, Edgar Nelson Sánchez Camperos, Carlos Daniel Acosta Medina, Germán Augusto Castellanos Domínguez
• Format: Journal Article
• Language: English
• Published: Universidad de Antioquia 28-02-2013
• Subjects: dimensionality reduction; PCA; PPCA; WPCA; WRDA
• ISSN: 0120-6230; 2422-2844
• DOI: 10.17533/udea.redin.14674

We propose to use a Fisher type discriminant objective function addressed to weighted principal component analysis (WPCA) and weighted regularized discriminant analysis (WRDA) for dimensionality reduction. Additionally, two different proofs for the convergence of the method are obtained. First one analytically, by using the completeness theorem, and second one algebraically, employing spectral decomposition. The objective function depends on two parameters U matrix being the rotation and D diagonal matrix weight of relevant features, respectively. These parameters are computed iteratively, in order to maximize the reduction. Relevant features were obtained by determining the eigenvector associated to the most weighted eigenvalue onthe maximum value in U. Performance evaluation of the reduction methods was carried out on 70 benchmark databases. Results showed that weighted reduction methods presented the best behavior, PCA and PPCA lower than 17% while WPCA and WRDA higher than 45%. Particularly, WRDA method had the best performance in the 75% of the cases compared with the others studied here.