Autoencoding beyond pixels using a learned similarity metric

Anders Boesen Lindbo Larsen; Søren Kaae Sønderby; Hugo Larochelle; Ole Winther

Autoencoding beyond pixels using a learned similarity metric

Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, Ole Winther

Computational and RNA Biology

183 Citations (Scopus)

Abstract

We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder (VAE) with a generative adversarial network (GAN) we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we replace element-wise errors with feature-wise errors to better capture the data distribution while offering invariance towards e.g. translation. We apply our method to images of faces and show that it outperforms VAEs with element-wise similarity measures in terms of visual fidelity. Moreover, we show that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic.

Original language	English
Title of host publication	Proceedings of The 33rd International Conference on Machine Learning
Editors	Maria Florina Balcan, Kilian Q. Weinberger
Number of pages	9
Publication date	2016
Pages	1558–1566
ISBN (Electronic)	978-151082900-8
Publication status	Published - 2016
Event	33rd International Conference on Machine Learning - New York, United States Duration: 19 Jun 2016 → 24 Jun 2016 Conference number: 33

Conference

Conference	33rd International Conference on Machine Learning
Number	33
Country/Territory	United States
City	New York
Period	19/06/2016 → 24/06/2016

Series	JMLR: Workshop and Conference Proceedings
Volume	48

Access to Document

http://www.jmlr.org/proceedings/papers/v48/larsen16.html

Cite this

Autoencoding beyond pixels using a learned similarity metric. / Larsen, Anders Boesen Lindbo; Sønderby, Søren Kaae; Larochelle, Hugo et al.
Proceedings of The 33rd International Conference on Machine Learning. ed. / Maria Florina Balcan; Kilian Q. Weinberger. 2016. p. 1558–1566 (JMLR: Workshop and Conference Proceedings, Vol. 48).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

Larsen, ABL, Sønderby, SK, Larochelle, H & Winther, O 2016, Autoencoding beyond pixels using a learned similarity metric. in MF Balcan & KQ Weinberger (eds), Proceedings of The 33rd International Conference on Machine Learning. JMLR: Workshop and Conference Proceedings, vol. 48, pp. 1558–1566, 33rd International Conference on Machine Learning, New York, United States, 19/06/2016. <http://www.jmlr.org/proceedings/papers/v48/larsen16.html>

@inproceedings{bacf45f293f24c06989d89fa70a84267,

title = "Autoencoding beyond pixels using a learned similarity metric",

abstract = "We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder (VAE) with a generative adversarial network (GAN) we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we replace element-wise errors with feature-wise errors to better capture the data distribution while offering invariance towards e.g. translation. We apply our method to images of faces and show that it outperforms VAEs with element-wise similarity measures in terms of visual fidelity. Moreover, we show that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic.",

author = "Larsen, {Anders Boesen Lindbo} and S{\o}nderby, {S{\o}ren Kaae} and Hugo Larochelle and Ole Winther",

year = "2016",

language = "English",

series = "JMLR: Workshop and Conference Proceedings",

publisher = "Microtome Publishing",

pages = "1558–1566",

editor = "Balcan, {Maria Florina} and Weinberger, {Kilian Q.}",

booktitle = "Proceedings of The 33rd International Conference on Machine Learning",

note = "33rd International Conference on Machine Learning ; Conference date: 19-06-2016 Through 24-06-2016",

}

TY - GEN

T1 - Autoencoding beyond pixels using a learned similarity metric

AU - Larsen, Anders Boesen Lindbo

AU - Sønderby, Søren Kaae

AU - Larochelle, Hugo

AU - Winther, Ole

N1 - Conference code: 33

PY - 2016

Y1 - 2016

N2 - We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder (VAE) with a generative adversarial network (GAN) we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we replace element-wise errors with feature-wise errors to better capture the data distribution while offering invariance towards e.g. translation. We apply our method to images of faces and show that it outperforms VAEs with element-wise similarity measures in terms of visual fidelity. Moreover, we show that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic.

AB - We present an autoencoder that leverages learned representations to better measure similarities in data space. By combining a variational autoencoder (VAE) with a generative adversarial network (GAN) we can use learned feature representations in the GAN discriminator as basis for the VAE reconstruction objective. Thereby, we replace element-wise errors with feature-wise errors to better capture the data distribution while offering invariance towards e.g. translation. We apply our method to images of faces and show that it outperforms VAEs with element-wise similarity measures in terms of visual fidelity. Moreover, we show that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic.

M3 - Article in proceedings

AN - SCOPUS:84999041243

T3 - JMLR: Workshop and Conference Proceedings

SP - 1558

EP - 1566

BT - Proceedings of The 33rd International Conference on Machine Learning

A2 - Balcan, Maria Florina

A2 - Weinberger, Kilian Q.

T2 - 33rd International Conference on Machine Learning

Y2 - 19 June 2016 through 24 June 2016

ER -

Autoencoding beyond pixels using a learned similarity metric

Abstract

Conference

Access to Document

Fingerprint

Cite this