6 ms·

Just tried this with CGPT4: Me: Which weighs more, a pound of bricks or a kilogram of feathers? ChatGPT: A kilogram weighs more than a pound. Specifically, 1

Just tried this with CGPT4: Me: Which weighs more, a pound of bricks or a kilogram of feathers? ChatGPT: A kilogram weighs more than a pound. Specifically, 1 kilogram is approximately 2.20462 pounds. So, a kilogram of feathers weighs more than a pound of bricks.

pmart123 3y ago
I don't believe the outputs are completely deterministic?
- zzzzzzzza 3y ago
  depends on the temperature, but overall I think parent commenter has less understanding than gpt4 lol (referring to braindead "if I can produce a single counterexample of logical inference failing, clearly it has no understanding" take). literally don't even waste time debating takes like this when you could be talking to gpt4 instead.
godelski 3y ago
Ask it if it is correct. Do you have a clear and well defined stopping criteria? I jokingly wrote "in before GPT4" because our example is the most simple of cases. I've seen GPT4 make similar mistakes, just not as blatant ones. Which in some ways is better but in other ways worse. I should answer that I've been asking this exact question at minimum 3 times a month since ChatGPT came out. It should be in their training. Especially as I've tweeted and discussed with plenty of LLM people about it. The point is not the specific question (while an egregious example), but rather how much you can trust the system.
- Ajedi32 3y ago
  > The point is not the specific question (while an egregious example), but rather how much you can trust the system. And the answer is "you can trust the system a little (enough to answer that logic puzzle correctly anyway), and you'll likely be able to trust future versions more". Obviously nobody is expecting LLMs to be able to fix every possible bug, but it's entirely possible they will be able to fix enough to be useful. Saying "LLMs will not achieve this. Full stop." seems premature.
  - godelski 3y ago
    I have to say it because a lot of HN users try to convince me that GPT has understanding. We actually have 0 ML models that demonstrate understanding. Frankly, we don't know if this is even possible yet. Sure, in the future I believe we'll have AGI, but an AGI would likely recognize that my statement is contextualized around the current environment and fitting common human speech patters rather than making an absolute immutable statement.
    - silverbax88 3y ago
      They are just stochastic parrots. Impressive and useful, but that's all they are.
      - Timon3 3y ago
        The comment above is a completely original thought. It was not memorized and pattern-matched.
    - Ajedi32 3y ago
      Depends on how you define "understanding" I guess, but it certainly seems to me like GPT-4 "understands" the concept of weight well enough to not be tripped up by this particular logic puzzle. You can argue it's just an imitation of "understanding" and not the "real thing", but how good does an imitation have to be before it's functionally identical to the real thing? You can argue different cherry-picked examples seem to demonstrate a lack of understanding, but does that mean it doesn't possess understanding in general or just that it doesn't understand those particular topics? I agree GPT-4 is not a human-like AGI, and that the people expecting it to behave as one or expecting a future version right around the corner that does are likely in for disappointment. But at the same time, LLMs are capable of writing coherent, never-before-seen code from natural language instructions, solving basic never-before-seen natural language logic puzzles, and playing half-decent chess moves from never-before-seen positions[1], all in a single model that was never specifically trained to do any of those things. To say those tasks don't require at least some level of intelligence or "understanding" seems far fetched to me. [1]: https://dkb.blog/p/chatgpts-chess-elo-is-1400 https://dkb.blog/p/chatgpts-chess-elo-is-1400

pmart123 3y ago

I don't believe the outputs are completely deterministic?

zzzzzzzza 3y ago
depends on the temperature, but overall I think parent commenter has less understanding than gpt4 lol (referring to braindead "if I can produce a single counterexample of logical inference failing, clearly it has no understanding" take). literally don't even waste time debating takes like this when you could be talking to gpt4 instead.

godelski 3y ago

Ask it if it is correct. Do you have a clear and well defined stopping criteria? I jokingly wrote "in before GPT4" because our example is the most simple of cases. I've seen GPT4 make similar mistakes, just not as blatant ones. Which in some ways is better but in other ways worse. I should answer that I've been asking this exact question at minimum 3 times a month since ChatGPT came out. It should be in their training. Especially as I've tweeted and discussed with plenty of LLM people about it. The point is not the specific question (while an egregious example), but rather how much you can trust the system.

Ajedi32 3y ago
> The point is not the specific question (while an egregious example), but rather how much you can trust the system. And the answer is "you can trust the system a little (enough to answer that logic puzzle correctly anyway), and you'll likely be able to trust future versions more". Obviously nobody is expecting LLMs to be able to fix every possible bug, but it's entirely possible they will be able to fix enough to be useful. Saying "LLMs will not achieve this. Full stop." seems premature.
- godelski 3y ago
  I have to say it because a lot of HN users try to convince me that GPT has understanding. We actually have 0 ML models that demonstrate understanding. Frankly, we don't know if this is even possible yet. Sure, in the future I believe we'll have AGI, but an AGI would likely recognize that my statement is contextualized around the current environment and fitting common human speech patters rather than making an absolute immutable statement.
  - silverbax88 3y ago
    They are just stochastic parrots. Impressive and useful, but that's all they are.
    - Timon3 3y ago
      The comment above is a completely original thought. It was not memorized and pattern-matched.
  - Ajedi32 3y ago
    Depends on how you define "understanding" I guess, but it certainly seems to me like GPT-4 "understands" the concept of weight well enough to not be tripped up by this particular logic puzzle. You can argue it's just an imitation of "understanding" and not the "real thing", but how good does an imitation have to be before it's functionally identical to the real thing? You can argue different cherry-picked examples seem to demonstrate a lack of understanding, but does that mean it doesn't possess understanding in general or just that it doesn't understand those particular topics? I agree GPT-4 is not a human-like AGI, and that the people expecting it to behave as one or expecting a future version right around the corner that does are likely in for disappointment. But at the same time, LLMs are capable of writing coherent, never-before-seen code from natural language instructions, solving basic never-before-seen natural language logic puzzles, and playing half-decent chess moves from never-before-seen positions[1], all in a single model that was never specifically trained to do any of those things. To say those tasks don't require at least some level of intelligence or "understanding" seems far fetched to me. [1]: https://dkb.blog/p/chatgpts-chess-elo-is-1400 https://dkb.blog/p/chatgpts-chess-elo-is-1400