Resilient Backpropagation (Rprop) for Batch-learning in TensorFlow

Ciprian Florescu; Christian Igel

Resilient Backpropagation (Rprop) for Batch-learning in TensorFlow

1 Citation (Scopus)

Abstract

The resilient backpropagation (Rprop) algorithms are fast and accurate batch learning methods for neural networks. We describe their implementation in the popular machine learning framework TensorFlow. We present the first empirical evaluation of Rprop for training recurrent neural networks with gated recurrent units. In our experiments, Rprop with default hyperparameters outperformed vanilla steepest descent as well as the optimization algorithms RMSprop and Adam even if their hyperparameters were tuned.

Original language	English
Publication date	2018
Number of pages	5
Publication status	Published - 2018
Event	International Conference on Learning Representations: Workshop - Vancouver, Canada Duration: 30 Apr 2018 → 3 May 2018 Conference number: 6 https://iclr.cc/

Workshop

Workshop	International Conference on Learning Representations
Number	6
Country/Territory	Canada
City	Vancouver
Period	30/04/2018 → 03/05/2018
Internet address	https://iclr.cc/

Cite this

@conference{6fad1eceaefa4284b3c06d6b9aff0aeb,

title = "Resilient Backpropagation (Rprop) for Batch-learning in TensorFlow",

abstract = "The resilient backpropagation (Rprop) algorithms are fast and accurate batch learning methods for neural networks. We describe their implementation in the popular machine learning framework TensorFlow. We present the first empirical evaluation of Rprop for training recurrent neural networks with gated recurrent units. In our experiments, Rprop with default hyperparameters outperformed vanilla steepest descent as well as the optimization algorithms RMSprop and Adam even if their hyperparameters were tuned.",

author = "Ciprian Florescu and Christian Igel",

year = "2018",

language = "English",

note = "International Conference on Learning Representations : Workshop, ICLR ; Conference date: 30-04-2018 Through 03-05-2018",

url = "https://iclr.cc/",

}

TY - CONF

T1 - Resilient Backpropagation (Rprop) for Batch-learning in TensorFlow

AU - Florescu, Ciprian

AU - Igel, Christian

N1 - Conference code: 6

PY - 2018

Y1 - 2018

N2 - The resilient backpropagation (Rprop) algorithms are fast and accurate batch learning methods for neural networks. We describe their implementation in the popular machine learning framework TensorFlow. We present the first empirical evaluation of Rprop for training recurrent neural networks with gated recurrent units. In our experiments, Rprop with default hyperparameters outperformed vanilla steepest descent as well as the optimization algorithms RMSprop and Adam even if their hyperparameters were tuned.

AB - The resilient backpropagation (Rprop) algorithms are fast and accurate batch learning methods for neural networks. We describe their implementation in the popular machine learning framework TensorFlow. We present the first empirical evaluation of Rprop for training recurrent neural networks with gated recurrent units. In our experiments, Rprop with default hyperparameters outperformed vanilla steepest descent as well as the optimization algorithms RMSprop and Adam even if their hyperparameters were tuned.

UR - https://openreview.net/forum?id=r1R0o7yDz

M3 - Paper

T2 - International Conference on Learning Representations

Y2 - 30 April 2018 through 3 May 2018

ER -

Resilient Backpropagation (Rprop) for Batch-learning in TensorFlow

Abstract

Workshop

Other files and links

Fingerprint

Cite this