A comparison of LSTM and GRU networks for learning symbolic sequences

Cahuantzi, Roberto and Chen, Xinye and Güttel, Stefan (2021) A comparison of LSTM and GRU networks for learning symbolic sequences. [MIMS Preprint]

Text A_comparison_of_LSTM_and_GRU_networks_forlearning_symbolic_sequences__Paper_.pdf Download (1MB)

Abstract

We explore relations between the hyper-parameters of a recurrent neural network (RNN) and the complexity of string sequences it is able to memorize. We compare long short-term memory (LSTM) networks and gated recurrent units (GRUs). We find that an increase of RNN depth does not necessarily result in better memorization capability when the training time is constrained. Our results also indicate that the learning rate and the number of units per layer are among the most important hyper-parameters to be tuned. Generally, GRUs outperform LSTM networks on low complexity sequences while on high complexity sequences LSTMs perform better.

Item Type:	MIMS Preprint
Subjects:	MSC 2010, the AMS's Mathematics Subject Classification > 68 Computer science
Depositing User:	Stefan Güttel
Date Deposited:	03 Jul 2021 11:58
Last Modified:	03 Jul 2021 11:58
URI:	https://eprints.maths.manchester.ac.uk/id/eprint/2825

Actions (login required)

View Item