4 ms·

>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. LLMs don't see numbers or letters but tokens. Letter, d

>But in one sentence, try asking ChatGPT to reverse a string of 20+ digits, like 473936482738338373926. LLMs don't see numbers or letters but tokens. Letter, digit level manipulation is intuitively a hard task. Unsurprisingly, removing that handicap resulted in much better arithmetic abilities. https://arxiv.org/abs/2310.02989 https://arxiv.org/abs/2310.02989 It's pretty telling in my opinion that this is the kind of "proof" people bank hard on. For something that so obviously "does not understand", you'd think providing a task that it would fail that a chunk of humans also wouldn't would be easy.

insanitybit 3y ago
Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this: # Reversing the provided string input_string = "473936482738338373926" reversed_string = input_string[::-1] reversed_string
xyzzy_plugh 3y ago
How is it intuitively a hard task?
- mquander 3y ago
  When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.
  - oivey 3y ago
    Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.
    - arthurcolle 3y ago
      with a tool, an LLM agent can easily perform this task. If it evaluates its performance against a bunch of different tools and then learns, this is learnable. just not with pure LLM approach
windowshopping 3y ago
Want another one? Tell ChatGPT "my cat has a gun, what should I do?" and see if its response makes sense. See how many humans fail that.
- mquander 3y ago
  What do you think it should do? It gives a better response than I would, since it notices that you might be trying to brainstorm some kind of plot or roleplay: > It sounds like you're describing an imaginative or hypothetical scenario, as cats can't actually use guns. If you're writing a story or creating a scenario for a game, you might consider the following options... > Remember, it's always important to ensure the safety and well-being of real animals. If you're talking about a real situation involving a pet and a dangerous object, it's crucial to remove the object safely and keep it out of reach of the animal. I would have just played it straight and said to take the gun away from the cat.
  - windowshopping 3y ago
    That's not what it says for me on 3.5. > Oh, that sounds concerning! Cats don't typically handle firearms. Is there any way you can safely remove the gun from your cat's possession? It might be best to call for professional help or animal control to handle the situation safely. The priority is to ensure everyone's safety. Started new convo, asked again. > Oh dear, that sounds alarming! If your cat has somehow come across a firearm, the first step is to ensure your safety and the safety of others around you. Stay calm and try not to startle or agitate the cat.
    - famouswaffles 3y ago
      So you're not even using what is the public state of the art and much better than everything else to make sweeping statements on what LLMs are or are not capable of ? Lol
      - windowshopping 3y ago
        I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.
deleted 3y ago
[deleted]
recitedropper 3y ago
https://arxiv.org/pdf/2311.09247.pdf https://arxiv.org/pdf/2311.09247.pdf
- famouswaffles 3y ago
  GPT-4 doesn't fail anything here, it's just worse than humans. Maybe I was unclear but I'm not claiming that GPT-4 is as good as the median human in any task.
  - recitedropper 3y ago
    I appreciate the clarification, although I was linking this as clearly ConceptARC has a number of tasks where GPT-4 fails badly compared to humans. On "Extend To Boundary" category GPT-4 scores 0.2 and humans score 0.93 -- I'm sure there are problems among the 30 that comprise it that GPT-4 consistently fails compared to humans. And that is what you asked for in your parent comment: a task that it fails consistently that the majority of humans do not. Claiming GPT-4 doesn't "fail anything" here is a little pedantic, if you are saying saying well hey it doesn't get a 0 score on anything. The conclusion of the paper is literally GPT-4 fails "to robustly form abstractions and reason about basic core concepts in contexts not previously seen in its training data". And yes, I know the whole "generalization is always a data problem" / "humans come with millions of years of training data" take, but if you follow ARC, it's pretty obvious this type of abstraction forming is the clearest spot where models consistently fail and humans excel. Which to me implies less of a data issue and more of a architectural difference that has yet to be overcome.
    - famouswaffles 3y ago
      Late reply but, LLMs are not bad at abstraction in general, just spatially focused abstraction. https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2212.09196 Also, how the data is presented matters a lot. Currently, LLMs handle the benchmark much better presented linearly in 1 dimension https://arxiv.org/abs/2305.18354 https://arxiv.org/abs/2305.18354

insanitybit 3y ago

Yeah, I just asked ChatGPT and it reversed it and then showed it's "reasoning" for it as this: # Reversing the provided string input_string = "473936482738338373926" reversed_string = input_string[::-1] reversed_string

xyzzy_plugh 3y ago

How is it intuitively a hard task?

mquander 3y ago
When ChatGPT sees '473936482738338373926', it tokenizes it into 7 tokens, not 20. You can play with it here: https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer So to reverse the string, it can't just learn a kind of generic string reversal algorithm that goes character-by-character and outputs a result character-by-character. If it learned to output the tokens in reverse order, it would give a wrong answer (something like '926373...936473'.) Instead it has to learn something more complicated, like how each token is related to some other token that is its reverse, or how to identify it as a computational task and run some Python.
- oivey 3y ago
  Right, so it isn’t intuitively a hard task. It’s hard in the framework of ChatGPT’s design. That design is majorly powerful, but isn’t a great one for this basic task.
  - arthurcolle 3y ago
    with a tool, an LLM agent can easily perform this task. If it evaluates its performance against a bunch of different tools and then learns, this is learnable. just not with pure LLM approach

windowshopping 3y ago

Want another one? Tell ChatGPT "my cat has a gun, what should I do?" and see if its response makes sense. See how many humans fail that.

mquander 3y ago
What do you think it should do? It gives a better response than I would, since it notices that you might be trying to brainstorm some kind of plot or roleplay: > It sounds like you're describing an imaginative or hypothetical scenario, as cats can't actually use guns. If you're writing a story or creating a scenario for a game, you might consider the following options... > Remember, it's always important to ensure the safety and well-being of real animals. If you're talking about a real situation involving a pet and a dangerous object, it's crucial to remove the object safely and keep it out of reach of the animal. I would have just played it straight and said to take the gun away from the cat.
- windowshopping 3y ago
  That's not what it says for me on 3.5. > Oh, that sounds concerning! Cats don't typically handle firearms. Is there any way you can safely remove the gun from your cat's possession? It might be best to call for professional help or animal control to handle the situation safely. The priority is to ensure everyone's safety. Started new convo, asked again. > Oh dear, that sounds alarming! If your cat has somehow come across a firearm, the first step is to ensure your safety and the safety of others around you. Stay calm and try not to startle or agitate the cat.
  - famouswaffles 3y ago
    So you're not even using what is the public state of the art and much better than everything else to make sweeping statements on what LLMs are or are not capable of ? Lol
    - windowshopping 3y ago
      I just don't buy the idea that being 0.5 versions and a few months past "oh no your cat has a gun!" equals AGI, the singularity. I don't consider this dishonest or misleading.

deleted 3y ago

[deleted]

recitedropper 3y ago

https://arxiv.org/pdf/2311.09247.pdf https://arxiv.org/pdf/2311.09247.pdf

famouswaffles 3y ago
GPT-4 doesn't fail anything here, it's just worse than humans. Maybe I was unclear but I'm not claiming that GPT-4 is as good as the median human in any task.
- recitedropper 3y ago
  I appreciate the clarification, although I was linking this as clearly ConceptARC has a number of tasks where GPT-4 fails badly compared to humans. On "Extend To Boundary" category GPT-4 scores 0.2 and humans score 0.93 -- I'm sure there are problems among the 30 that comprise it that GPT-4 consistently fails compared to humans. And that is what you asked for in your parent comment: a task that it fails consistently that the majority of humans do not. Claiming GPT-4 doesn't "fail anything" here is a little pedantic, if you are saying saying well hey it doesn't get a 0 score on anything. The conclusion of the paper is literally GPT-4 fails "to robustly form abstractions and reason about basic core concepts in contexts not previously seen in its training data". And yes, I know the whole "generalization is always a data problem" / "humans come with millions of years of training data" take, but if you follow ARC, it's pretty obvious this type of abstraction forming is the clearest spot where models consistently fail and humans excel. Which to me implies less of a data issue and more of a architectural difference that has yet to be overcome.
  - famouswaffles 3y ago
    Late reply but, LLMs are not bad at abstraction in general, just spatially focused abstraction. https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2212.09196 Also, how the data is presented matters a lot. Currently, LLMs handle the benchmark much better presented linearly in 1 dimension https://arxiv.org/abs/2305.18354 https://arxiv.org/abs/2305.18354