Support vector machine ensembles for discriminant analysis for ranking principal components

Abstract

The problem of ranking linear subspaces in principal component analysis (PCA), for multiclass classification tasks, has been addressed by building support vector machine (SVM) ensembles and AdaBoost.M2 technique. This methodology, named multi-class discriminant principal components analysis (Multi-Class.M2 DPCA), is motivated by the fact that the first PCA components do not necessarily represent important discriminant directions to separate sample groups. The Multi-Class.M2 DPCA proposal presents fundamental issues related to the weakening methodology, parametrization, strategy for SVM bias, and classification versus reconstruction performance. Also, it is observed a lack of comparisons betweenMulti-Class.M2DPCAandfeatureweightingtechniques.Motivatedbythesefacts, this paper firstly presents a unified formulation to generate weakened SVM approaches and to derive different strategies of the literature. These strategies are analyzed within MultiClass.M2 DPCA methodology and its parametrization to realize the best one for ranking PCA features in face image analysis. Moreover, this work proposes variants to improve that Multi-Class.M2 DPCA configuration using strategies that incorporate SVM bias and sensitivity analysis results. The obtained Multi-Class.M2 DPCA setups are applied in the computational experiments for both classification and reconstruction problems. The results showthatMulti-Class.M2DPCAachieveshigherrecognitionratesusinglessPCAfeatures, as well as robust reconstruction and interpretation of the data.