机器学习算法介绍及相关参考文献

Linear Discriminant AnalysisLinear Discriminant Analysis (LDA) is a linear inherently multi-class cl

Linear Discriminant Analysis Linear Discriminant Analysis (LDA) is alinear inherently multi-class classification method. It was originally introduced by Fisher for two classes [9], but was later extended for multiple classes by Rao ) W gx Wx T ( [26]. In particular, LDA computes aclassification function where is selected as — , t _| | WS W W — B the linear projection that maximizes the Fisher-criterion where arg max , opt T || WS W W W W S and are the within-class and the between class scatter matrices (see, e.g., [7]). The corresponding S WB optimal solution for this optimization problem is given by the solution of the generalized S S 1 _ eigenproblem or directly by computing the eigenvectors for 九 — Sw Sw .Since the rank WB BW SS -w bb 】 of is bounded by the rank of there are S c — non-zero eigenvalues resulting in a 1 WtX r c e — — (1) 1 L c—xn ([-dimensional subspace ,which GE {1 x m ° preserves the most discriminant information. For classification of anew sample the class label ，…， } c is assigned according to the result of anearest neighbor classification. For that purpose, the Euclidean d v —Wt gx 卩 distances of the projected sample and the class centers in the LDA space are compared: () ii —arg min ((), ) . w dg xv i <i <c 1 Loog et al. [19] showed that for more than two classes maximizing the Fisher criterion in Eq. (7) provides only asuboptimal solution! In particular, optimizing the Fisher criterion provides an optimal solution with respect to the Bayes error for two classes, but this can not be generalized for multiple classes. Nevertheless LDA can be applied for many practical multi-class problems. This was also confirmed by theoretical considerations by Mart'mthey showed that increasing the number of classes decreases the ez and Zhu [20]. However, separability. [9] R. A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179—188, 1936. [26] C. R. Rao. The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society —Series B, 10(2):159-203, 1948. [7] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification.John Wiley &Sons, 2000. [19] M. Loog, R. P. W. Duin, and R. Haeb-Umbach. Multiclass linear dimension reduction by weighted pairwise fisher criteria. IEEE Trans. PAMI, 23(7):762- 766, 2001. [20] A. M. Mart'inez and M. Zhu. Where are linear feature extraction methods applicable? IEEE Trans. PAMI, 27(12):1934-1944, 2005.