4 ms·

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that mod

This is an interesting thought exercise! The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. Another interesting bit of experiment people did when the GPT-4 class models launched were to test out spatial awareness. For eg, you could describe with words a construction made of blocks, spheres and so on and then ask questions about the stability of the structure. The newer models more often than not got it right. Of course, that hasn't stopped people from making tired old claims of "stochastic parrot" and so forth but I think if your model can solve such reasoning problem (spatial or otherwise) then there's something really interesting to discover. I'm glad that folks at Anthropic are really trying to create a better understanding of these models (https://www.anthropic.com/research/mapping-mind-language-model https://www.anthropic.com/research/mapping-mind-language-mod...)

amelius 2y ago
> The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. But this is merely a definition of what it means to "understand" something. For example just tabulating many input/output combinations would not follow this definition.
- prasoonds 2y ago
  You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning. Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e., they claim that no such world model exists and that the model has rote memorized these reasoning scenarios.
  - zarzavat 2y ago
    Why not just give it a series of directions e.g. go one step forwards, turn right, etc like a LOGO program, and then ask it if it is back to the start or not. If the series of instructions is randomly generated then this is impossible to solve without a world model.
    - prasoonds 2y ago
      People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks. Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM. However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.
- dTal 2y ago
  From the preface to the 20th anniversary edition of Godel, Escher, Bach: "Meaning cannot be kept out of formal systems when sufficiently complex isomorphisms arise. Meaning comes in despite one's best efforts to keep symbols meaningless! ...When a system of "meaningless" symbols has patterns in it that accurately track, or mirror, various phenomena in the world, then that tracking or mirroring imbues the symbols with some degree of meaning -- indeed, such tracking or mirroring is no less and no more than what meaning is. Depending on how complex and subtle and reliable the tracking is, different degrees of meaningfulness arise." In other words, when one can reliably ask a language model a question and get a sensible answer, one is forced to conclude that it does in some sense "understand" what it is saying. This is also I think the essential philosophical thrust of the Turing Test, which is often misunderstood as a mere benchmark. (I notice a common objection to examples of LLMs clearly demonstrating understanding is "it saw something similar in the training set". That may be true (though unfalsifiable) in any given instance, but the number of permutations of things LLMs correctly respond to far exceed the size of any training set. They are certainly generalizing, and interpreting their inputs on a conceptual level.)
  - mensetmanusman 2y ago
    It’s a type of understanding, but one we are not really familiar with, because it seems to understand tasks that it was trained on, but fails and other very basic ways on simple but different tasks.
nalrcnG 2y ago
But where do yo get access to these premium models? I only have free access to CoPilot (which claimed at one point to be GPT4) and it cannot solve any of these issues reliably. It spectacularly fails at slight variations of the goat/cabbage/lion/river problem, it cannot solve spatial or mathematical questions reliably. Do some of the researchers that hype up ChatGPT get access to a special version? I'm not inclined to buy a subscription to find out and the AI Reddits aren't that positive either.
- seanhunter 2y ago
  How people get access to the premium models is typically to sign up[1] for an account and pay a subscription. GPT-4o is significantly different from original GPT-4 which is again very significantly different from GPT-3.5. This is all moving fast. So if you don't want to pay for a subscription I think you can get some free use of anthropic's most capable model (Opus) - I don't know the status of what you can get for free from openai. The opinion of AI reddit is only really going to get you so far because the capabilities of the models are wildly different for different use cases, so you really need to be able to try it out for yourself and see if it can do what you need it to do. [1] Somewhere like https://openai.com/chatgpt/pricing/ https://openai.com/chatgpt/pricing/ https://claude.ai/ https://claude.ai/ or similar
- prasoonds 2y ago
  Like the sibling comment says, you typically have to pay for the premium model (and believe me, they are miles ahead of the free stuff - free copilot uses gpt3.5 which is a full 2 generations behind). The other way is to get an API key from OpenAI or Anthropic and then, you can pay per token which is incredibly cheap - load it up with $5 and it can easily last you months if you're not using it much. As a dev, I cannot recommend these better models enough. The ROI on my $10-$20 that I spend every month is easily in hundreds to thousands of dollars.
namaria 2y ago
Stochastic parrot is way too simplistic a dismissal. It's hand wavy. But I don't get what's so impressive about the nuanced language of a language model that has been given datacenter amounts of compute and virtually all of written word ever put digitalized. Yeah it's the first actually functioning natural language interface. At what cost though. It's completely out of proportion with the benefits and only bubble level 'investing' can justify this.
- lxgr 2y ago
  Are you referring to the (one-time) training cost or the cost per inferred token? The latter is pretty acceptable these days, especially with smaller models.
  - namaria 2y ago
    Small models can't do impressive language corner cases. And the benefits of the models that can scale linearly while their cost grows geometrically. Also claiming training costs are 'one off' when there's so much resources being poured into training new and bigger models is disingenuous.
    - mensetmanusman 2y ago
      The resources pouring in still pale in comparison to capital expenditures in the semi conductor industry.
      - namaria 2y ago
        The inflation in semi conductor capital expenditure is the problem caused by the AI hype.
        mensetmanusman 2y ago
        Nah, tsmc spending tens of billion annually on capital before OpenAI.

amelius 2y ago

> The theory goes - that if the model "understands" this scenario, then internally, it has has created something that models the real world. But this is merely a definition of what it means to "understand" something. For example just tabulating many input/output combinations would not follow this definition.

prasoonds 2y ago
You're right of course. Let me rephrase. What I was trying to say was: If the model solves some reasoning tasks that may imply it's building some inner world model which is fundamental to reasoning. That in-turn might mean that the model is reasoning. Others have argued against the "inner world model" theory and suggested that solving reasoning tasks is merely an extension of the "stochastic parrot" scenario - i.e., they claim that no such world model exists and that the model has rote memorized these reasoning scenarios.
- zarzavat 2y ago
  Why not just give it a series of directions e.g. go one step forwards, turn right, etc like a LOGO program, and then ask it if it is back to the start or not. If the series of instructions is randomly generated then this is impossible to solve without a world model.
  - prasoonds 2y ago
    People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks. Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM. However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.
dTal 2y ago
From the preface to the 20th anniversary edition of Godel, Escher, Bach: "Meaning cannot be kept out of formal systems when sufficiently complex isomorphisms arise. Meaning comes in despite one's best efforts to keep symbols meaningless! ...When a system of "meaningless" symbols has patterns in it that accurately track, or mirror, various phenomena in the world, then that tracking or mirroring imbues the symbols with some degree of meaning -- indeed, such tracking or mirroring is no less and no more than what meaning is. Depending on how complex and subtle and reliable the tracking is, different degrees of meaningfulness arise." In other words, when one can reliably ask a language model a question and get a sensible answer, one is forced to conclude that it does in some sense "understand" what it is saying. This is also I think the essential philosophical thrust of the Turing Test, which is often misunderstood as a mere benchmark. (I notice a common objection to examples of LLMs clearly demonstrating understanding is "it saw something similar in the training set". That may be true (though unfalsifiable) in any given instance, but the number of permutations of things LLMs correctly respond to far exceed the size of any training set. They are certainly generalizing, and interpreting their inputs on a conceptual level.)
- mensetmanusman 2y ago
  It’s a type of understanding, but one we are not really familiar with, because it seems to understand tasks that it was trained on, but fails and other very basic ways on simple but different tasks.

nalrcnG 2y ago

But where do yo get access to these premium models? I only have free access to CoPilot (which claimed at one point to be GPT4) and it cannot solve any of these issues reliably. It spectacularly fails at slight variations of the goat/cabbage/lion/river problem, it cannot solve spatial or mathematical questions reliably. Do some of the researchers that hype up ChatGPT get access to a special version? I'm not inclined to buy a subscription to find out and the AI Reddits aren't that positive either.

seanhunter 2y ago
How people get access to the premium models is typically to sign up[1] for an account and pay a subscription. GPT-4o is significantly different from original GPT-4 which is again very significantly different from GPT-3.5. This is all moving fast. So if you don't want to pay for a subscription I think you can get some free use of anthropic's most capable model (Opus) - I don't know the status of what you can get for free from openai. The opinion of AI reddit is only really going to get you so far because the capabilities of the models are wildly different for different use cases, so you really need to be able to try it out for yourself and see if it can do what you need it to do. [1] Somewhere like https://openai.com/chatgpt/pricing/ https://openai.com/chatgpt/pricing/ https://claude.ai/ https://claude.ai/ or similar
prasoonds 2y ago
Like the sibling comment says, you typically have to pay for the premium model (and believe me, they are miles ahead of the free stuff - free copilot uses gpt3.5 which is a full 2 generations behind). The other way is to get an API key from OpenAI or Anthropic and then, you can pay per token which is incredibly cheap - load it up with $5 and it can easily last you months if you're not using it much. As a dev, I cannot recommend these better models enough. The ROI on my $10-$20 that I spend every month is easily in hundreds to thousands of dollars.

namaria 2y ago

Stochastic parrot is way too simplistic a dismissal. It's hand wavy. But I don't get what's so impressive about the nuanced language of a language model that has been given datacenter amounts of compute and virtually all of written word ever put digitalized. Yeah it's the first actually functioning natural language interface. At what cost though. It's completely out of proportion with the benefits and only bubble level 'investing' can justify this.

lxgr 2y ago
Are you referring to the (one-time) training cost or the cost per inferred token? The latter is pretty acceptable these days, especially with smaller models.
- namaria 2y ago
  Small models can't do impressive language corner cases. And the benefits of the models that can scale linearly while their cost grows geometrically. Also claiming training costs are 'one off' when there's so much resources being poured into training new and bigger models is disingenuous.
  - mensetmanusman 2y ago
    The resources pouring in still pale in comparison to capital expenditures in the semi conductor industry.
    - namaria 2y ago
      The inflation in semi conductor capital expenditure is the problem caused by the AI hype.
      - mensetmanusman 2y ago
        Nah, tsmc spending tens of billion annually on capital before OpenAI.