Evolution of Stacked Autoencoders

Tim Silhan; Stefan Oehmcke; Oliver Kramer

doi:10.1109/CEC.2019.8790182

Evolution of Stacked Autoencoders

Tim Silhan, Stefan Oehmcke^*, Oliver Kramer

^*Corresponding author af dette arbejde

2 Citationer (Scopus)

Abstract

Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.

Originalsprog	Engelsk
Titel	2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Proceedings
Antal sider	8
Forlag	Institute of Electrical and Electronics Engineers Inc.
Publikationsdato	jun. 2019
Sider	823-830
Artikelnummer	8790182
ISBN (Elektronisk)	9781728121536
DOI	https://doi.org/10.1109/CEC.2019.8790182
Status	Udgivet - jun. 2019
Begivenhed	2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Wellington, New Zealand Varighed: 10 jun. 2019 → 13 jun. 2019

Konference

Konference	2019 IEEE Congress on Evolutionary Computation, CEC 2019
Land/Område	New Zealand
By	Wellington
Periode	10/06/2019 → 13/06/2019
Sponsor	et al., Facebook, IEEE, IEEE CIS, Tourism New Zealand, Victoria University of Wellington

Adgang til dokumentet

10.1109/CEC.2019.8790182

Andre filer og links

Link to publication in Scopus

Citationsformater

@inproceedings{23e782d33f924a5eae57f46b24bb3088,

title = "Evolution of Stacked Autoencoders",

abstract = "Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.",

keywords = "autoencoder, hyperparameter tuning, neuroevolution",

author = "Tim Silhan and Stefan Oehmcke and Oliver Kramer",

year = "2019",

month = jun,

doi = "10.1109/CEC.2019.8790182",

language = "English",

pages = "823--830",

booktitle = "2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "2019 IEEE Congress on Evolutionary Computation, CEC 2019 ; Conference date: 10-06-2019 Through 13-06-2019",

}

TY - GEN

T1 - Evolution of Stacked Autoencoders

AU - Silhan, Tim

AU - Oehmcke, Stefan

AU - Kramer, Oliver

PY - 2019/6

Y1 - 2019/6

N2 - Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.

AB - Choosing the best hyperparameters for neural networks is a big challenge. This paper proposes a method that automatically initializes and adjusts hyperparameters during the training process of stacked autoencoders. A population of autoencoders is trained with gradient-descent-based weight updates, while hyperparameters are mutated and weights are inherited in a Lamarckian kind of way. The training is conducted layer-wise, while each new layer initiates a new neuroevolutionary optimization process. In the fitness function of the evolutionary approach a dimensionality reduction quality measure is employed. Experiments show the contribution of the most significant hyperparameters, while analyzing their lineage during the training process. The results confirm that the proposed method outperforms a baseline approach on MNIST, FashionMNIST, and the Year Prediction Million Song Database.

KW - autoencoder

KW - hyperparameter tuning

KW - neuroevolution

UR - http://www.scopus.com/inward/record.url?scp=85071314505&partnerID=8YFLogxK

U2 - 10.1109/CEC.2019.8790182

DO - 10.1109/CEC.2019.8790182

M3 - Article in proceedings

SP - 823

EP - 830

BT - 2019 IEEE Congress on Evolutionary Computation, CEC 2019 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE Congress on Evolutionary Computation, CEC 2019

Y2 - 10 June 2019 through 13 June 2019

ER -

Evolution of Stacked Autoencoders

Abstract

Konference

Adgang til dokumentet

Andre filer og links

Fingeraftryk

Citationsformater