机器学习算法介绍及相关参考文献

LinearDiscriminantAnalysisLinearDiscriminantAnalysis(LDA)isalinearinherentlymulti-classclassificatio

LinearDiscriminantAnalysis LinearDiscriminantAnalysis(LDA)isalinearinherentlymulti-classclassificationmethod.Itwasoriginallyintroduced byFisherfortwoclasses[9],butwaslaterextendedformultipleclassesbyRao W gxWx T [26].Inparticular,LDAcomputesaclassificationfunctionwhereisselectedas , = () t _|W| WS W arg thelinearprojectionthatmaximizestheFisher-criterionwhere , = max opt T |WSW| W W S S andarethewithin-classandthebetweenclassscattermatrices(see,e.g.,[7]).The WB S correspondingoptimalsolutionforthisoptimizationproblemisgivenbythesolutionofthegeneralizedeigenproblem = 九 c SSSS Sw t ordirectlybycomputingtheeigenvectorsforJ.Sincetherankofisboundedbytherankoftherearenon-zeroei — ^1 genvaluesresultingina wbb WtXr c e 1Lcn (_1)x — ()-dimensionalsubspace,whichpreservesthemostdiscriminantinformation.Forclassificationo = m x e fanewsampletheclasslabel 丘诫 e ， {1,… } d c isassignedaccordingtotheresultofanearestneighborclassification.Forthatpurpose,theEuclideandistancesofthepro W t gxv () 卩 jectedsampleandtheclasscentersintheLDAspacearecompared: — ii . — e argmind(g(x),v) i << 1ic Loogetal.[19]showedthatformorethantwoclassesmaximizingtheFishercriterioninEq.(7)providesonlyasuboptimals olution!Inparticular,optimizingtheFishercriterionprovidesanoptimalsolutionwithrespecttotheBayeserrorfortwocla sses,butthiscannotbegeneralizedformultipleclasses.NeverthelessLDAcanbeappliedformanypracticalmulti-classpr oblems.ThiswasalsoconfirmedbytheoreticalconsiderationsbyMart'theyshowedthatinc inezandZhu[20].However, reasingthenumberofclassesdecreasestheseparability. [9]R.A.Fisher.Theuseofmultiplemeasurementsintaxonomicproblems.AnnalsofEugenics,7:179—188,1936.[26]C.R.Rao.T heutilizationofmultiplemeasurementsinproblemsofbiologicalclassification.JournaloftheRoyalStatisticalSociety—SeriesB,