Feature Extraction (vs. Feature Selection)

The terms feature extraction and feature selection are used in the context of predictive data mining, when the goal is to find a good predictive model for some phenomenon of interest based on a large number of predictors. While feature selection methods (such as the Feature Selection and Variable Screening methods) will attempt to identify the best predictors among the (sometimes thousands of) available predictors, feature extraction techniques attempt to aggregate or combine the predictors in some way to extract the common information contained in them that is most useful for building the model. Typical methods for feature extraction are Factor Analysis and Principal Components Analysis, Correspondence Analysis, Multidimensional Scaling, Partial Least Squares methods, or singular value decomposition, as, for example, used in text mining.