4 ms·

I think this is a good place to ask: are there any tasks left that LLMs still fail significantly more often than average humans? Not "sometimes it writes buggy

I think this is a good place to ask: are there any tasks left that LLMs still fail significantly more often than average humans? Not "sometimes it writes buggy code", or "it refuses to discuss ethics", but getting something wrong in a way that people on the street wouldn't. So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that mentions its own word count"). But those feel unsatisfactory, like asking a human to decode ASCII. I've personally seen some impressive feats from simple LLMs, like writing sensible code in a new language based only on a few surrounding lines. That shows a level of expertise beyond the common line "it's just mixing sentences from the training data".

dartos 3y ago
Creative writing. LLMs tend to overly stick to tropes or create plot holes Complex problem solving. There are only so many confounding variables that next token generation can handle Anything requiring recent knowledge. RAG exists, but in context learning only gets you so far.
- jameshart 3y ago
  The average person on the street sticks to tropes and creates plot holes, too.
- famouswaffles 3y ago
  He asked for a comparison with the average human. Even with the "helpful assistant" RLHF hampering GPT-4's fiction writing abilities, i do not think it is significantly worse than average human.
Jensson 3y ago
> So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that includes the count of words in itself"). But those feel unsatisfactory because those are not cognitive limitations. It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. Humans plan things based around their own limitations, if an AI model can't even do that how can it be considered an AGI?
- famouswaffles 3y ago
  >It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. I don't think Humans understand their limitations very well at all. We plan around limitations we observe, not necessarily limitations that are present. There are many surprising limitations only recently uncovered by advances in neuroscience. I'm not sure humans would qualify as self aware by your criteria. >if an AI model can't even do that how can it be considered an AGI? By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym.
  - Jensson 3y ago
    > By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym. I said they need to be aware of their own limitations, that is very well defined. Humans might not be aware of all our limitations, but we are aware of a lot of them and that lets us work around them. Working around your own limitations is necessary for AGI, otherwise there are many trivial problems you can't solve. It is possible that you can make many human crafted strategies and that goes far enough, but I doubt it since trying to make human crafted examples never has resulted in AGI advancements just allowing the bot to solve a specific problem well.
    - famouswaffles 3y ago
      >I said they need to be aware of their own limitations, that is very well defined. You seemed to imply that this "awareness" was some inherent ability of humans but it's very clearly not. If it were, we wouldn't be a lot more aware of these limitations today than even a few decades past. and if all the records we kept vanished in an instant today then the next generations of humans would be a lot less aware. This "awareness" you are talking about is little more than accumulated knowledge. By the realistic definition of 'aware of what is observed', i fail to see how LLMs don't demonstrate this.
- BoppreH 3y ago
  > it doesn't understand its own limitations, ie it has no self awareness. And how much self awareness do humans have, exactly? We walk around with heads filled with false memories[1], accidentally ignore surprisingly prominent stimuli[2], and struggle to understand things as simple as material color[3]. And people are mostly unaware of these limitations, or those examples would not be considered surprising! I think we plan around the limitations we observe, not our real limitations. If that's the case, then LLMs simply lack training data to notice those limitations. [1] https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory [2] https://www.youtube.com/watch?v=vJG698U2Mvo https://www.youtube.com/watch?v=vJG698U2Mvo (old selective attention test) [3] https://en.wikipedia.org/wiki/The_dress https://en.wikipedia.org/wiki/The_dress
foobarqux 3y ago
There are lots of examples, the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. I don't know how you can use these systems for any significant length of time and not run into logical errors.
- famouswaffles 3y ago
  >the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. That's not what that paper says
  - foobarqux 3y ago
    Yes, we discussed this before. You are wrong. If the system can produce the statement A is the mother of B but not B is son/daughter of A then it fails to do something humans can trivially do.
    - famouswaffles 3y ago
      The system can produce both and the paper makes this clear for those who bothered to read it.
      - foobarqux 3y ago
        No, you yourself said it can only do so in-context.
- BoppreH 3y ago
  But at different rates from humans? I ran into the exact same issue myself today, while studying German flash cards. I could remember e.g., "ruhig -> quiet", but not "quiet -> ruhig".
  - foobarqux 3y ago
    You are confusing memory with logical inference. "A is the mother of B" is basically the same logical concept as "B is the child of A", you wouldn't know one and not the other. Because LLMs are basically next-word-prediction systems they need to be trained on both pieces of text. In fact you could train them on logically inconsistent text like "A is the mother of B" and "B is not the child of A" and they would happily spit out nonsense.
snewman 3y ago
LLMs can't do just about any extended, context-dependent task, which is why they're far away from being drop-in replacements for people. Absolutely they're doing more than just regurgitating training data, but there's a long distance from there to "any task a human could do".
famouswaffles 3y ago
The real biggest thing so far is agency. You can make LLMs agentic but the current state of the art is still behind median human ability in this area. I think this will fall too but that's the current state.
recursivecaveat 3y ago
They used to be quite bad at common-sense-reasoning type tasks like HellaSwag: https://arxiv.org/pdf/1905.07830.pdf https://arxiv.org/pdf/1905.07830.pdf but it seems like they're creeping up into the high-80s for % of responses correct now.

dartos 3y ago

Creative writing. LLMs tend to overly stick to tropes or create plot holes Complex problem solving. There are only so many confounding variables that next token generation can handle Anything requiring recent knowledge. RAG exists, but in context learning only gets you so far.

jameshart 3y ago
The average person on the street sticks to tropes and creates plot holes, too.
famouswaffles 3y ago
He asked for a comparison with the average human. Even with the "helpful assistant" RLHF hampering GPT-4's fiction writing abilities, i do not think it is significantly worse than average human.

Jensson 3y ago

> So far I've only seen examples based on tokenization ("what's the third letter in word?"), and the fact that answers are generated sequentially ("write a sentence that includes the count of words in itself"). But those feel unsatisfactory because those are not cognitive limitations. It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. Humans plan things based around their own limitations, if an AI model can't even do that how can it be considered an AGI?

famouswaffles 3y ago
>It could solve those, the reason it can't is that it doesn't understand its own limitations, ie it has no self awareness. I don't think Humans understand their limitations very well at all. We plan around limitations we observe, not necessarily limitations that are present. There are many surprising limitations only recently uncovered by advances in neuroscience. I'm not sure humans would qualify as self aware by your criteria. >if an AI model can't even do that how can it be considered an AGI? By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym.
- Jensson 3y ago
  > By being artificial and generally intelligent. I don't see anything about "self-awareness" (ill-defined as it may be) anywhere in that acronym. I said they need to be aware of their own limitations, that is very well defined. Humans might not be aware of all our limitations, but we are aware of a lot of them and that lets us work around them. Working around your own limitations is necessary for AGI, otherwise there are many trivial problems you can't solve. It is possible that you can make many human crafted strategies and that goes far enough, but I doubt it since trying to make human crafted examples never has resulted in AGI advancements just allowing the bot to solve a specific problem well.
  - famouswaffles 3y ago
    >I said they need to be aware of their own limitations, that is very well defined. You seemed to imply that this "awareness" was some inherent ability of humans but it's very clearly not. If it were, we wouldn't be a lot more aware of these limitations today than even a few decades past. and if all the records we kept vanished in an instant today then the next generations of humans would be a lot less aware. This "awareness" you are talking about is little more than accumulated knowledge. By the realistic definition of 'aware of what is observed', i fail to see how LLMs don't demonstrate this.
BoppreH 3y ago
> it doesn't understand its own limitations, ie it has no self awareness. And how much self awareness do humans have, exactly? We walk around with heads filled with false memories[1], accidentally ignore surprisingly prominent stimuli[2], and struggle to understand things as simple as material color[3]. And people are mostly unaware of these limitations, or those examples would not be considered surprising! I think we plan around the limitations we observe, not our real limitations. If that's the case, then LLMs simply lack training data to notice those limitations. [1] https://en.wikipedia.org/wiki/False_memory https://en.wikipedia.org/wiki/False_memory [2] https://www.youtube.com/watch?v=vJG698U2Mvo https://www.youtube.com/watch?v=vJG698U2Mvo (old selective attention test) [3] https://en.wikipedia.org/wiki/The_dress https://en.wikipedia.org/wiki/The_dress

foobarqux 3y ago

There are lots of examples, the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. I don't know how you can use these systems for any significant length of time and not run into logical errors.

famouswaffles 3y ago
>the recent paper on LLMs failing to invert logic (A is B therefore B is A) is one. That's not what that paper says
- foobarqux 3y ago
  Yes, we discussed this before. You are wrong. If the system can produce the statement A is the mother of B but not B is son/daughter of A then it fails to do something humans can trivially do.
  - famouswaffles 3y ago
    The system can produce both and the paper makes this clear for those who bothered to read it.
    - foobarqux 3y ago
      No, you yourself said it can only do so in-context.
BoppreH 3y ago
But at different rates from humans? I ran into the exact same issue myself today, while studying German flash cards. I could remember e.g., "ruhig -> quiet", but not "quiet -> ruhig".
- foobarqux 3y ago
  You are confusing memory with logical inference. "A is the mother of B" is basically the same logical concept as "B is the child of A", you wouldn't know one and not the other. Because LLMs are basically next-word-prediction systems they need to be trained on both pieces of text. In fact you could train them on logically inconsistent text like "A is the mother of B" and "B is not the child of A" and they would happily spit out nonsense.

snewman 3y ago

LLMs can't do just about any extended, context-dependent task, which is why they're far away from being drop-in replacements for people. Absolutely they're doing more than just regurgitating training data, but there's a long distance from there to "any task a human could do".

famouswaffles 3y ago

The real biggest thing so far is agency. You can make LLMs agentic but the current state of the art is still behind median human ability in this area. I think this will fall too but that's the current state.

recursivecaveat 3y ago

They used to be quite bad at common-sense-reasoning type tasks like HellaSwag: https://arxiv.org/pdf/1905.07830.pdf https://arxiv.org/pdf/1905.07830.pdf but it seems like they're creeping up into the high-80s for % of responses correct now.