"Curse" of Dimensionality

The term curse of dimensionality (Bellman, 1961, Bishop, 1995) generally refers to the difficulties involved in fitting models, estimating parameters, or optimizing a function in many dimensions, usually in the context of neural networks. As the dimensionality of the input data space (i.e., the number of predictors) increases, it becomes exponentially more difficult to find global optima for the parameter space, i.e., to fit models. In practice, the complexity of neural networks becomes unmanageable when the number of inputs into the neural network exceeds a few hundreds or even less, depending on the complexity of the respective neural network architecture. Hence, it is simply a practical necessity to pre-screen and preselect from among a large set of input (predictor) variables those that are of likely utility for predicting the outputs (dependent variables) of interest.

The STATISTICA Feature Selection and Variable Screening module is a unique tool for selecting continuous and/or categorical predictors for regression or classification tasks, using methods that do not bias the selection in favor of any particular model or technique for predictive data mining that might be applied to the selection.