A note on generalization loss when evolving adaptive pattern recognition systems

TY - JOUR

T1 - A note on generalization loss when evolving adaptive pattern recognition systems

AU - Igel, Christian

PY - 2013

Y1 - 2013

N2 - Evolutionary computing provides powerful methods for designing pattern recognition systems. This design process is typically based on finite sample data and therefore bears the risk of overfitting. This paper aims at raising the awareness of various types of overfitting and at providing guidelines for how to deal with them. We restrict our considerations to the predominant scenario in which fitness computations are based on point estimates. Three different sources of losing generalization performance when evolving learning machines, namely overfitting to training, test, and final selection data, are identified, discussed, and experimentally demonstrated. The importance of a pristine hold-out data set for the selection of the final result from the evolved candidates is highlighted. It is shown that it may be beneficial to restrict this last selection process to a subset of the evolved candidates.

AB - Evolutionary computing provides powerful methods for designing pattern recognition systems. This design process is typically based on finite sample data and therefore bears the risk of overfitting. This paper aims at raising the awareness of various types of overfitting and at providing guidelines for how to deal with them. We restrict our considerations to the predominant scenario in which fitness computations are based on point estimates. Three different sources of losing generalization performance when evolving learning machines, namely overfitting to training, test, and final selection data, are identified, discussed, and experimentally demonstrated. The importance of a pristine hold-out data set for the selection of the final result from the evolved candidates is highlighted. It is shown that it may be beneficial to restrict this last selection process to a subset of the evolved candidates.

U2 - 10.1109/TEVC.2012.2197214

DO - 10.1109/TEVC.2012.2197214

M3 - Journal article

SN - 1089-778X

VL - 17

SP - 345

EP - 352

JO - I E E E Transactions on Evolutionary Computation

JF - I E E E Transactions on Evolutionary Computation

IS - 3

ER -