5 ms·

I did provide a counter argument and referenced evidence "backing it up": Namely LLMs can learn languages that are not human-compatible (Moro, "Secrets of Words

I did provide a counter argument and referenced evidence "backing it up": Namely LLMs can learn languages that are not human-compatible (Moro, "Secrets of Words"). I seriously doubt you have read, never mind understood, any of the papers you cite; if you had we could discuss what's wrong with them. Without even having read them there are obvious problems like the fact that you can't measure a large number of individual neuron activation in the brain and that different deep learning networks have substantially different activations so they cannot be meaningfully be similar to the brain if they aren’t even similar to one another (or that the similarity is so nebulous as to be meaningless). But this is moot because as I said at the outset there are fundamental differences at the functional level which make existing LLMs (and deep learning) unlike brains.

chaxor 3y ago
I think we may agree more than you realize, but perhaps we aren't clear about what we mean. As I said pretty early on - they are similar at some level. To make an analogy, that could be as simple as a submarine (w2v) and a fish (authentic language) now 'uses fins' to move (bert). They're now more similar, but not the same. Of course there are differences, but we can learn by studying what structure has been added and how in order to work towards better theory (as well as looking at what differences there are). I have read all of these studies, as I am an academic in related fields, and you are correct that the resolution is not as good as we would hope (it's better in the ventral visual stream in non human primates, which is one reason we started there); however it doesn't negate the importance of the predictive power here. We can make very useful tools out of these models which can improve the lives of non-verbal people for example. Another interesting set of works going in recently is in testing out the poverty of stimulus. There is now evidence from many labs that LMs learn with the same amount of data as humans, so the sample efficiency arguments and the 'poverty of stimulus' aren't really as successful as arguments for training difference either. To the point of humans not learning certain languages, I haven't seen it proven that humans are absolutely incapable of learning certain languages. I do know that children ignore much of sequence structure and create their own internal structures, but this does not negate learning languages which have some arbitrary set of rules. I'm aware that models learn DNA more easily _practically_, but this is more of a statement about human experience and various other environmental factors - not a mathematical proof that it's impossible for humans to learn. Furthermore, that is a nice property of these systems. If they are capable of modeling many different phenomenon, they are useful for many things. If they can settle on a model configuration that linearly maps to a similar activation space with similar topological properties to a patient, that's also very useful. In other words, If the topological properties of the activation space are more constrained in the cortex vs more flexible NNs, but the NNs can fit to the constrained space, that doesn't seem like an insurmountable problem in studying either trained system. Ultimately, I think we can agree that there are differences, but there are also striking similarities that can be very useful for improving science and medicine going forward, and perhaps (with _substantial_ effort in linguistics and mechanistic interpretability fields) we may be able to improve some of our understanding of linguistics and/or neuroscience (or perhaps not, but it's at least a promising potential lead).
- foobarqux 3y ago
  > I haven't seen it proven that humans are absolutely incapable of learning certain languages. I cited Moro who showed it in a series of experiments. > they are useful many things The fact that they are much more general than humans makes them not restrictive enough to be models of the human language faculty.

chaxor 3y ago

I think we may agree more than you realize, but perhaps we aren't clear about what we mean. As I said pretty early on - they are similar at some level. To make an analogy, that could be as simple as a submarine (w2v) and a fish (authentic language) now 'uses fins' to move (bert). They're now more similar, but not the same. Of course there are differences, but we can learn by studying what structure has been added and how in order to work towards better theory (as well as looking at what differences there are). I have read all of these studies, as I am an academic in related fields, and you are correct that the resolution is not as good as we would hope (it's better in the ventral visual stream in non human primates, which is one reason we started there); however it doesn't negate the importance of the predictive power here. We can make very useful tools out of these models which can improve the lives of non-verbal people for example. Another interesting set of works going in recently is in testing out the poverty of stimulus. There is now evidence from many labs that LMs learn with the same amount of data as humans, so the sample efficiency arguments and the 'poverty of stimulus' aren't really as successful as arguments for training difference either. To the point of humans not learning certain languages, I haven't seen it proven that humans are absolutely incapable of learning certain languages. I do know that children ignore much of sequence structure and create their own internal structures, but this does not negate learning languages which have some arbitrary set of rules. I'm aware that models learn DNA more easily _practically_, but this is more of a statement about human experience and various other environmental factors - not a mathematical proof that it's impossible for humans to learn. Furthermore, that is a nice property of these systems. If they are capable of modeling many different phenomenon, they are useful for many things. If they can settle on a model configuration that linearly maps to a similar activation space with similar topological properties to a patient, that's also very useful. In other words, If the topological properties of the activation space are more constrained in the cortex vs more flexible NNs, but the NNs can fit to the constrained space, that doesn't seem like an insurmountable problem in studying either trained system. Ultimately, I think we can agree that there are differences, but there are also striking similarities that can be very useful for improving science and medicine going forward, and perhaps (with _substantial_ effort in linguistics and mechanistic interpretability fields) we may be able to improve some of our understanding of linguistics and/or neuroscience (or perhaps not, but it's at least a promising potential lead).

foobarqux 3y ago
> I haven't seen it proven that humans are absolutely incapable of learning certain languages. I cited Moro who showed it in a series of experiments. > they are useful many things The fact that they are much more general than humans makes them not restrictive enough to be models of the human language faculty.