3 ms·

Because Chomsky is trying to build a bird, and Norvig is trying to build an airplane. It's much easier to build an airplane to fly than a bird. Chomsky is tryi

by mempko 3y ago

Because Chomsky is trying to build a bird, and Norvig is trying to build an airplane. It's much easier to build an airplane to fly than a bird. Chomsky is trying to explain how humans create language. LLM are creating language, but not the way humans do. Nothing about this paper refutes Chomsky's claims.

williamcotton 3y ago
Chomsky has been adding parameters to his theory to handle exceptions in a way that mimics the endless series of conditional statements appended to knowledge systems of yore. In neuroscience, predictive processing has gained immense favor and can explain language in ways that have nothing to do with innate grammar. https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding Exactly how well did "building a bird" work for building flying machines? Birds use the same principle as a fixed wing when it comes to soaring flight. "Building a bird" without the principles of an airfoil and just mimicking the flapping wings does not result in flight.
- bbor 3y ago
  …he has? Isn’t his modern term “minimalism”, where he tries to simplify things as much as possible? Regardless, continuing to study the field in no way implies that he’s backed down or meaningfully evolved his basic theories of Universal Grammars. He’s very much still confident in them. Re: predictive processing, in what way does that relate to language…? Even if you apply it to language in a way not mentioned in the linked article at all, I don’t see even the rough shape of how it would refute (/be mutually exclusive with) generative grammars. Maybe I’m just missing something because I don’t know much neuro? Re: building a bird… yeah that’s their point, you don't try to build a bird, you try to study birds. Chomsky cares about what we are, not building machines to do our drudgery. I don’t think I agree entirely with that singular focus, but you see the appeal, no?
  - williamcotton 3y ago
    Gina Kuperberg at Harvard has done extensive work on how predictive coding operates in language processing. Her lab has explored how people generate predictions during language comprehension and how this is reflected in neural activity. https://projects.iq.harvard.edu/kuperberglab/people/gina-r-kuperberg https://projects.iq.harvard.edu/kuperberglab/people/gina-r-k... Here's some relevant research: https://projects.iq.harvard.edu/kuperberglab/publications/dynamic-predictive-coding-across-left-fronto-temporal-language-hierarchy https://projects.iq.harvard.edu/kuperberglab/publications/dy... I'll point out that neuroscientists have yet to find the "generative grammar" part of the brain but have seen evidence of a very large network of neurons...
  - williamcotton 3y ago
    Re: building a bird… I didn’t come up with the analogy and if we all want to stop talking about building birds I would be the first to agree. But I can’t help by think about all of the whacky ideas like antigravity vital forces that biologists contrived to explain how birds could fly and that it took Bernoulli and the rigorous study of those principles that led to the airfoil… which is how birds actually soar through the air. BTW, what the fuck happened to these forums in the last few years? It seems like most people base their opinions on how opposite they are to Sam Altman and Elon Musk’s as opposed to any geeky principles of discovery. I highly doubt that most of y’all would have been ardent supporters of generative grammar five years ago… but slap the word LLM on something and boy howdy! It’s kind of nice, I learn quite a bit defending good ideas. All y’all get is fake internet points. I fully understand any and all downvotes! Have fun!
    - bbor 3y ago
      You got me, I believe in the most cited living academic’s theories on cognition to stick it to Elon Musk. So glad you used your geeky principles of discovery to divine that one…
- gizmo686 3y ago
  Chomsky is not trying to build a flying machine. He is trying to study birds. Biologists don't look at a 737 and say "that's obviously the way flight works, I wonder where the engine on that seagull is".
  - williamcotton 3y ago
    The only reason I was talking about "building a bird" in the first place is because that is what the person I was responding to was talking about. At this point I don't even understand what this analogy is supposed to be. Is the bird supposed to be "language"? Or "humans"? Or "human language"? What's the point of this study, just to chit-chat about things? What proof do we have the Chomsky's theories of language are true? Neuroscience has actual research that backs the claims of predictive coding and language acquisition and comprehension! Some names in case you're curious: Gina Kuperberg, Ellen Lau, Florian Jaeger and Roger Levy, just to name a few of the more influential researchers.
- spookie 3y ago
  What are these parameters Chomsky has been adding? I'm interested, not to be taken as some defensive remark or the like.
  - williamcotton 3y ago
    He starts with the original Transformational-generative grammar in the late 50s, and then as more and more exceptions and complexities of language were found that didn't fit neatly into the transformational-generative model, the theory was extended to account for these exceptions, creating the Extended Standard Theory (syntactic constraints, X-bar theory), later revised to Revised Extended Standard Theory (restrictions upon X-bar theory, complementizer). Then we've got the addition of the Principles and Parameters theory (head-initial, head-final) in the 80s, which led to the Minimalist program as things had started to get out of hand. https://en.wikipedia.org/wiki/Generative_grammar https://en.wikipedia.org/wiki/Generative_grammar https://en.wikipedia.org/wiki/Transformational_grammar https://en.wikipedia.org/wiki/Transformational_grammar https://en.wikipedia.org/wiki/X-bar_theory https://en.wikipedia.org/wiki/X-bar_theory https://en.wikipedia.org/wiki/Principles_and_parameters https://en.wikipedia.org/wiki/Principles_and_parameters https://en.wikipedia.org/wiki/Government_and_binding_theory https://en.wikipedia.org/wiki/Government_and_binding_theory https://en.wikipedia.org/wiki/Minimalist_program https://en.wikipedia.org/wiki/Minimalist_program
throwawaymaths 3y ago
> It's much easier to build an airplane to fly than a bird. Not really. You could probably put together a flapping bird toy (can buy these mass manufactured too) in about a couple of months of trial and error. Not quite as sophisticated as feathers but the principles are the same. You probably couldn't build an airplane.

by mempko 3y ago

williamcotton 3y ago

Chomsky has been adding parameters to his theory to handle exceptions in a way that mimics the endless series of conditional statements appended to knowledge systems of yore. In neuroscience, predictive processing has gained immense favor and can explain language in ways that have nothing to do with innate grammar. https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding Exactly how well did "building a bird" work for building flying machines? Birds use the same principle as a fixed wing when it comes to soaring flight. "Building a bird" without the principles of an airfoil and just mimicking the flapping wings does not result in flight.

bbor 3y ago
…he has? Isn’t his modern term “minimalism”, where he tries to simplify things as much as possible? Regardless, continuing to study the field in no way implies that he’s backed down or meaningfully evolved his basic theories of Universal Grammars. He’s very much still confident in them. Re: predictive processing, in what way does that relate to language…? Even if you apply it to language in a way not mentioned in the linked article at all, I don’t see even the rough shape of how it would refute (/be mutually exclusive with) generative grammars. Maybe I’m just missing something because I don’t know much neuro? Re: building a bird… yeah that’s their point, you don't try to build a bird, you try to study birds. Chomsky cares about what we are, not building machines to do our drudgery. I don’t think I agree entirely with that singular focus, but you see the appeal, no?
- williamcotton 3y ago
  Gina Kuperberg at Harvard has done extensive work on how predictive coding operates in language processing. Her lab has explored how people generate predictions during language comprehension and how this is reflected in neural activity. https://projects.iq.harvard.edu/kuperberglab/people/gina-r-kuperberg https://projects.iq.harvard.edu/kuperberglab/people/gina-r-k... Here's some relevant research: https://projects.iq.harvard.edu/kuperberglab/publications/dynamic-predictive-coding-across-left-fronto-temporal-language-hierarchy https://projects.iq.harvard.edu/kuperberglab/publications/dy... I'll point out that neuroscientists have yet to find the "generative grammar" part of the brain but have seen evidence of a very large network of neurons...
- williamcotton 3y ago
  Re: building a bird… I didn’t come up with the analogy and if we all want to stop talking about building birds I would be the first to agree. But I can’t help by think about all of the whacky ideas like antigravity vital forces that biologists contrived to explain how birds could fly and that it took Bernoulli and the rigorous study of those principles that led to the airfoil… which is how birds actually soar through the air. BTW, what the fuck happened to these forums in the last few years? It seems like most people base their opinions on how opposite they are to Sam Altman and Elon Musk’s as opposed to any geeky principles of discovery. I highly doubt that most of y’all would have been ardent supporters of generative grammar five years ago… but slap the word LLM on something and boy howdy! It’s kind of nice, I learn quite a bit defending good ideas. All y’all get is fake internet points. I fully understand any and all downvotes! Have fun!
  - bbor 3y ago
    You got me, I believe in the most cited living academic’s theories on cognition to stick it to Elon Musk. So glad you used your geeky principles of discovery to divine that one…
gizmo686 3y ago
Chomsky is not trying to build a flying machine. He is trying to study birds. Biologists don't look at a 737 and say "that's obviously the way flight works, I wonder where the engine on that seagull is".
- williamcotton 3y ago
  The only reason I was talking about "building a bird" in the first place is because that is what the person I was responding to was talking about. At this point I don't even understand what this analogy is supposed to be. Is the bird supposed to be "language"? Or "humans"? Or "human language"? What's the point of this study, just to chit-chat about things? What proof do we have the Chomsky's theories of language are true? Neuroscience has actual research that backs the claims of predictive coding and language acquisition and comprehension! Some names in case you're curious: Gina Kuperberg, Ellen Lau, Florian Jaeger and Roger Levy, just to name a few of the more influential researchers.
spookie 3y ago
What are these parameters Chomsky has been adding? I'm interested, not to be taken as some defensive remark or the like.
- williamcotton 3y ago
  He starts with the original Transformational-generative grammar in the late 50s, and then as more and more exceptions and complexities of language were found that didn't fit neatly into the transformational-generative model, the theory was extended to account for these exceptions, creating the Extended Standard Theory (syntactic constraints, X-bar theory), later revised to Revised Extended Standard Theory (restrictions upon X-bar theory, complementizer). Then we've got the addition of the Principles and Parameters theory (head-initial, head-final) in the 80s, which led to the Minimalist program as things had started to get out of hand. https://en.wikipedia.org/wiki/Generative_grammar https://en.wikipedia.org/wiki/Generative_grammar https://en.wikipedia.org/wiki/Transformational_grammar https://en.wikipedia.org/wiki/Transformational_grammar https://en.wikipedia.org/wiki/X-bar_theory https://en.wikipedia.org/wiki/X-bar_theory https://en.wikipedia.org/wiki/Principles_and_parameters https://en.wikipedia.org/wiki/Principles_and_parameters https://en.wikipedia.org/wiki/Government_and_binding_theory https://en.wikipedia.org/wiki/Government_and_binding_theory https://en.wikipedia.org/wiki/Minimalist_program https://en.wikipedia.org/wiki/Minimalist_program

throwawaymaths 3y ago

> It's much easier to build an airplane to fly than a bird. Not really. You could probably put together a flapping bird toy (can buy these mass manufactured too) in about a couple of months of trial and error. Not quite as sophisticated as feathers but the principles are the same. You probably couldn't build an airplane.