> The notion of innate grammers has been pretty much replaced with evidence that you get much better and more human like performance (and the ability to actuall

> The notion of innate grammers has been pretty much replaced with evidence that you get much better and more human like performance (and the ability to actually learn the bloody things) out of RNNs and the like. No not really. Barring concrete physiological evidence, innateness is simply an unfalsifiable position. I do think you can say that consensus among scientists might have shifted against innateness, but that is definitely not because of some computational model such as RNN with some passing resemblance to observed behavior of human neurons. The way these networks are configured and trained has nothing to do with real brains, nor does the abstraction in neural nets leave room for any electrochemical effects. > The interesting question is why RNNs are able to produce things that resemble formal languages at first pass. Is it? A neural network can achieve that by simulating a Turing machine or simpler automaton. I don't see what that brings to the table? That's just doing something we could already do, but less efficiently. I think the interesting thing is the opposite: if the RNN can do something which couldn't be done before with other models.