Technically, it's not two layers. It's only two layers of parameters. A RNN runs the two layers repeatedly at each sequential input. So if my input/output is li

Technically, it's not two layers. It's only two layers of parameters. A RNN runs the two layers repeatedly at each sequential input. So if my input/output is like '1000~>fizz', it can be seen as a feedforward NN with at least 8 layers. (This is how it's possible to train a RNN in the first place: it gets 'unrolled' over time and turned into a big NN.)