4 ms·

The experts in the field say we need a philosophical breakthrough. Isn't everyone else inexperienced in this regard? https://aeon.co/essays/how-close-are-we-to

The experts in the field say we need a philosophical breakthrough. Isn't everyone else inexperienced in this regard? https://aeon.co/essays/how-close-are-we-to-creating-artificial-intelligence https://aeon.co/essays/how-close-are-we-to-creating-artifici...

lostmsu 4y ago
> David Deutschis a physicist at the University of Oxford and a fellow of the Royal Society. What field were you referring to?
- cactusplant7374 4y ago
  It could be linguistics, philosophy, or knowing enough of the history of those fields that make one an expert. I think Chomsky's argument on AI and specifically cognition and the brain is quite useful. Yet, you never hear Altman or Carmack talking about cognition or how computers can understand the meaning of something like a human. They aren't interested in such questions. But to conduct an experiment don't you have to know what you are looking for? Does a chemist do experiments by mixing 1 million compounds at a time?
  - lostmsu 4y ago
    I generally have pretty low regard for philosophy, and consider Popper + current scientific method to be SoTA. Its relationship to the nature of cognition overall seems pretty dubious. As for linguistics, IMHO the existence and success of GPT pretty much puts Chomsky into the proven wrong bucket, so again, not a good example. (his whole point used to be that statistical model can't learn syntax in principle, and GPT's syntax is close to impeccable) Re: a chemist. Well sort of. Because technically speaking a molecule of the same compound in a certain location and with certain energy is different from another molecule in a different location and with different energy. And even if you disregard that, why would you think that doing 1 million compounds could not significantly move material sciences forward? It is not like they don't want to do that, it is more of that they can't in practice at this time.
    - cactusplant7374 4y ago
      > As for linguistics, IMHO the existence and success of GPT pretty much puts Chomsky into the proven wrong bucket, so again, not a good example. (his whole point used to be that statistical model can't learn syntax in principle, and GPT's syntax is close to impeccable) What do you disagree with? He appears to be correct. The software hasn’t learned anything. It mixes and matches based on training data. https://m.youtube.com/watch?v=ndwIZPBs8Y4 https://m.youtube.com/watch?v=ndwIZPBs8Y4
      - lostmsu 4y ago
        According to the scientific method, on which the rest of the natural sciences are currently based, GPT is a valid model of GPT's syntax. There are "alternatives" for the method according to some philosophers, but AFAIK none of them are useful to any degree and can be considered fringe at this point.
    - foobarqux 4y ago
      LLMs haven't "learned" syntax, that's the point. It doesn't matter if you just want to predict syntax (engineering) only if you want to understand the human language faculty (science) and nearly no one is interested in the latter.
      - lostmsu 4y ago
        The fact that you don't understand how GPT models language does not make it less of a model. It did learn the syntax, you are just incapable to grasp the formula it represents.
        foobarqux 4y ago
        > The fact that you don't understand how GPT models language does not make it less of a model. It did learn the syntax, you are just incapable to grasp the formula it represents. The whole point of science is understanding and LLMs don't provide understanding of how human language works.

lostmsu 4y ago

> David Deutschis a physicist at the University of Oxford and a fellow of the Royal Society. What field were you referring to?

cactusplant7374 4y ago
It could be linguistics, philosophy, or knowing enough of the history of those fields that make one an expert. I think Chomsky's argument on AI and specifically cognition and the brain is quite useful. Yet, you never hear Altman or Carmack talking about cognition or how computers can understand the meaning of something like a human. They aren't interested in such questions. But to conduct an experiment don't you have to know what you are looking for? Does a chemist do experiments by mixing 1 million compounds at a time?
- lostmsu 4y ago
  I generally have pretty low regard for philosophy, and consider Popper + current scientific method to be SoTA. Its relationship to the nature of cognition overall seems pretty dubious. As for linguistics, IMHO the existence and success of GPT pretty much puts Chomsky into the proven wrong bucket, so again, not a good example. (his whole point used to be that statistical model can't learn syntax in principle, and GPT's syntax is close to impeccable) Re: a chemist. Well sort of. Because technically speaking a molecule of the same compound in a certain location and with certain energy is different from another molecule in a different location and with different energy. And even if you disregard that, why would you think that doing 1 million compounds could not significantly move material sciences forward? It is not like they don't want to do that, it is more of that they can't in practice at this time.
  - cactusplant7374 4y ago
    > As for linguistics, IMHO the existence and success of GPT pretty much puts Chomsky into the proven wrong bucket, so again, not a good example. (his whole point used to be that statistical model can't learn syntax in principle, and GPT's syntax is close to impeccable) What do you disagree with? He appears to be correct. The software hasn’t learned anything. It mixes and matches based on training data. https://m.youtube.com/watch?v=ndwIZPBs8Y4 https://m.youtube.com/watch?v=ndwIZPBs8Y4
    - lostmsu 4y ago
      According to the scientific method, on which the rest of the natural sciences are currently based, GPT is a valid model of GPT's syntax. There are "alternatives" for the method according to some philosophers, but AFAIK none of them are useful to any degree and can be considered fringe at this point.
  - foobarqux 4y ago
    LLMs haven't "learned" syntax, that's the point. It doesn't matter if you just want to predict syntax (engineering) only if you want to understand the human language faculty (science) and nearly no one is interested in the latter.
    - lostmsu 4y ago
      The fact that you don't understand how GPT models language does not make it less of a model. It did learn the syntax, you are just incapable to grasp the formula it represents.
      - foobarqux 4y ago
        > The fact that you don't understand how GPT models language does not make it less of a model. It did learn the syntax, you are just incapable to grasp the formula it represents. The whole point of science is understanding and LLMs don't provide understanding of how human language works.