3 ms·

Well, again, I'm not sure what your prompts are, but here we go: https://chatgpt.com/c/7ead1985-83be-4dce-9fd7-29aafb248f01 https://chatgpt.com/c/7ead1985-83be

by stared 2y ago

Well, again, I'm not sure what your prompts are, but here we go: https://chatgpt.com/c/7ead1985-83be-4dce-9fd7-29aafb248f01 https://chatgpt.com/c/7ead1985-83be-4dce-9fd7-29aafb248f01 > The statistical model is confused by the superficial similarity between (a) and (b), just as Chomsky predicted decades ago. Well, WE are statistical models as well. So, any too-broad claims on the inability to understand natural language by ANY statistical models are falsified the moment they are spoken (or written); unless you go into Penrose-style Neoplatonism. --- Vide your doubt if I find a (counter)convincing example. Sure, any benchmark of human vs. model performance is an empirical verification. And yes, some artificial models may struggle with some tasks. For example, for the Winograd schema, there is a leap between GPT3 and GPT4: https://d-kz.medium.com/evaluating-gpt-3-and-gpt-4-on-the-winograd-schema-challenge-reasoning-test-e4de030d190d https://d-kz.medium.com/evaluating-gpt-3-and-gpt-4-on-the-wi.... What it's (to me) is ex-cathedra defining what is the English language. The actual natural language people use is full of utterances that are not correct (yet are easily understandable) and "correct forms" that are rarely used and, if so, misunderstood by anyone save for those who put conscious effort (linguists, lawyers, etc). For any discussions, it is essential to know if we are working with a real (and highly probabilistic) natural language or what of its concrete models (i.e., an abstraction).

foldr 2y ago
My prompts are in my comment. Your ChatGPT link sends me to a login screen. Humans (or English speakers at least) aren’t confused at all by the pair of sentences in my last comment. If you’re just going to try to deny the plain facts about how these sentences are (and aren’t) interpreted by English speakers then that’s really just a kind of grammatical flat-Earthism. The judgments at issue aren’t remotely subtle. > Well, WE are statistical models as well. This is begging the question. Chomsky would deny this.
- stared 2y ago
  So, I think that we differ at a fundamental level. While you prefer to work in a Neoplatonic world of ideas, I prefer empirical facts and convictions that all models are approximations. English grammar is not fixed per se; it evolves with region (have you ever been to Singapore?) and time. Your judgment (or Chomsky's, or anyone), however founded, is not a fact. It is an opinion up for experimental scrutiny. I don't say that your examples are incorrect. Still, measure the percentage of correct (or consistent) answers for humans against particular models. Otherwise, it might be maths, might be philosophy, might be arts, but it is not (empirical) science.
  - foldr 2y ago
    As far as I can see, your issue with Chomsky has nothing to do with the performance of modern LLMs. You just reject all the data that generative linguists take to be crucially informative as to the grammatical structure of natural languages. You would hold the same view if LLMs had never been invented. So it is really a common case of AI and cognitive science talking entirely at cross purposes. > English grammar is not fixed per se; it evolves with region (have you ever been to Singapore?) and time. Sure, but this is not the case for the examples I gave. There aren’t dialects of English where (b) has the interpretation that GPT-4o thinks it can have. It’s no use trying to muddy the empirical waters in the case of completely clear judgments about what English sentences can and can’t mean.

by stared 2y ago

foldr 2y ago

My prompts are in my comment. Your ChatGPT link sends me to a login screen. Humans (or English speakers at least) aren’t confused at all by the pair of sentences in my last comment. If you’re just going to try to deny the plain facts about how these sentences are (and aren’t) interpreted by English speakers then that’s really just a kind of grammatical flat-Earthism. The judgments at issue aren’t remotely subtle. > Well, WE are statistical models as well. This is begging the question. Chomsky would deny this.

stared 2y ago
So, I think that we differ at a fundamental level. While you prefer to work in a Neoplatonic world of ideas, I prefer empirical facts and convictions that all models are approximations. English grammar is not fixed per se; it evolves with region (have you ever been to Singapore?) and time. Your judgment (or Chomsky's, or anyone), however founded, is not a fact. It is an opinion up for experimental scrutiny. I don't say that your examples are incorrect. Still, measure the percentage of correct (or consistent) answers for humans against particular models. Otherwise, it might be maths, might be philosophy, might be arts, but it is not (empirical) science.
- foldr 2y ago
  As far as I can see, your issue with Chomsky has nothing to do with the performance of modern LLMs. You just reject all the data that generative linguists take to be crucially informative as to the grammatical structure of natural languages. You would hold the same view if LLMs had never been invented. So it is really a common case of AI and cognitive science talking entirely at cross purposes. > English grammar is not fixed per se; it evolves with region (have you ever been to Singapore?) and time. Sure, but this is not the case for the examples I gave. There aren’t dialects of English where (b) has the interpretation that GPT-4o thinks it can have. It’s no use trying to muddy the empirical waters in the case of completely clear judgments about what English sentences can and can’t mean.