I was just reading this paper: Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks Rahul Dey and Fathi M. Salem

It seems to me that perhaps the architecture of LSTMs and GRUs are overly complicated. And that the same problems could probably be solved with a simpler architecture.

I get the theory behind LSTMs and GRUs as they are kind of trying to model the short term memory. But really all the need to do is get rid of the gradient exploding problem of RNNs.

What is the latest research. Is there something simpler than a GRU?

Edit: Actually I found something called a MGU (minimal gated unit) which claims to be simpler. What's the latest?

  • 1
    $\begingroup$ Welcome to CS.SE! Can you edit the question to provide a full citation? (Authors, title, and where published) This is helpful if the link stops working, and also helps others find this page by search (e.g., by searching on the title of the paper). $\endgroup$ – D.W. Nov 15 '17 at 7:07
  • 2
    $\begingroup$ It would probably also help to expand all those acronyms. GRU is the Russian military intelligence agency, right? ;-) $\endgroup$ – David Richerby Nov 15 '17 at 11:06
  • $\begingroup$ Define "good" and "simple". $\endgroup$ – Raphael Nov 15 '17 at 16:25
  • $\begingroup$ What research have you done? Have you follow the references of that paper? $\endgroup$ – Raphael Nov 15 '17 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.