SANN - Automated Network Search (ANS) - Weight Decay Tab

Select the Weight decay tab of the SANN Automated Network Search (ANS) dialog box to specify the use of weight decay regularization. For information on the options that are common to all tabs, see SANN Automated Network Search (ANS).

This option encourages the development of smaller weights, which tends to reduce the problem of over-fitting, thereby potentially improving generalization performance of the network. Weight decay works by modifying the network's error function to penalize large weights - the result is an error function that compromises between performance and weight size. Consequently, too large a weight decay term may damage network performance unacceptably, and experimentation is generally needed to determine an appropriate weight decay factor for a particular problem domain.

Use weight decay (hidden layer). Select this option to apply weight decay regularization to the input-hidden layer weights.

Min/Max. Specify the minimum and maximum weight decay value for the hidden layer weights. This option is not available for RBF networks.

Use weight decay (output layer). Select this option to apply weight decay regularization to the hidden-output layer weights.

Min/Max. Specify the minimum and maximum weight decay value for the output layer weights.