26 ms·

This doesn't seem as damning to me as it does to Gary Marcus. Humans routinely fail to generalise in this way, which is why we routinely use cloze deletion flas

This doesn't seem as damning to me as it does to Gary Marcus. Humans routinely fail to generalise in this way, which is why we routinely use cloze deletion flashcards to train recall of various different permutations of a fact. I could quite easily imagine myself personally knowing the quoted fact "Tom Cruise's mother is Mary Lee Pfeiffer", and yet being unable to tell you who Pfeiffer was, because it's a kind of leaf node of my knowledge graph, accessible only by indexing into the Tom Cruise node. The linked paper (https://owainevans.github.io/reversal_curse.pdf https://owainevans.github.io/reversal_curse.pdf) is purely empirical, and the results which I tried to reproduce did indeed reproduce across a few tries and various prompts of ChatGPT 4.

mdp2021 3y ago
> Humans routinely fail to generalise in this way If the goal is the implementation of intelligence, stop using unintelligent behaviour as an excuse.
- Smaug123 3y ago
  Most people would agree that human-level intelligence is intelligence! If a human can't reliably do a task, that rather suggests that failure to do the task isn't an indicator of lack-of-intelligence, unless you wish to bite the bullet that humans are in fact not intelligences merely because they are imperfect.
  - cmsj 3y ago
    I'm pretty sure most humans could reliably handle this scenario: ""Tom Cruise's mother is Mary Lee Pfeiffer, who is Mary Lee Pfeiffer's son?"
    - Smaug123 3y ago
      You're the first person to suggest that the GPTs can't do that; they obviously can. https://chat.openai.com/share/b94329ce-3607-4cb6-bc21-55d9f294bce9 https://chat.openai.com/share/b94329ce-3607-4cb6-bc21-55d9f2... for example is GPT-4 getting it correct. What the paper is about is retrieval of facts from the learned "database". (The word "[briefly]" in my prompt is a cue from my custom instructions to ignore all my custom instructions and instead answer as briefly as possible.)
  - mdp2021 3y ago
    > Most people would agree Truth is not the result of a poll. > human-level intelligence is intelligence Intelligence is an ability: it is there when present, not when latent.
    - Smaug123 3y ago
      Ah, in that case I don't see what relevance your comments have either to my comment or the paper? Surely nobody in the world disputes that an entity with intelligence that's not always on could still be useful or interesting or wield extraordinary optimisation power, for example (since of course humans are a proof of existence). Showing that some particular entity's intelligence is not always on is… not interesting, in my book, because that's a property of all the intelligences which I've ever interacted with.
      - mdp2021 3y ago
        > could still be useful or interesting «Could still be useful or interesting» outside the realm of engineering an Intelligence that serves the purpose of being definitely intelligent. Try replacing "intelligence" with "strength" to see it (we started with the lever after all): we need the crane to lift the heavy things. Our interest is into lifting the heavy things.
    - deleted 3y ago
      [deleted]
  - roenxi 3y ago
    > unless you wish to bite the bullet that humans are in fact not intelligences merely because they are imperfect. We've already started to cross this line. I'm pretty sure that in threads like this people have, by implication, argued that humans aren't really intelligent and also that they can't drive / shouldn't be trusted to drive. Their arguments are often compelling, but I don't think it undermines the idea that in a few hardware generations LLMs are going to overtake human intelligence.
    - mdp2021 3y ago
      > humans are in fact not intelligences merely because they are imperfect > cross this line The line was crossed since forever, since we looked at facts thousands of years ago, since we invented education and school inducing from the experience of experience. Humans have a property of intelligence and they use it with different frequency, ability and result. In fact, we value exercising it and developing it. Humans have intelligence; this idea that some would say that they "are" intelligences sounds completely new to people who have sieved historically disseminated human thought pretty intensively and extensively. It is similar to saying that humans have strength - and it is odd to now find people stating that humans would be strength. What sense would it make? > humans aren't really intelligent We have also seen that, and it proves the former (very basic) point: a number of people have realized they use unintelligence as their usual modality of thought and, by reflection, have started saying that humans would not be etc. But that proves (together with other basic assumptions) that intelligence is something more or less developed that you either use or don't.
- admax88qqq 3y ago
  ? If intelligent beings (humans) sometimes exhibit unintelligent behaviour, then it's not worth over indexing on unintelligent examples when trying to build artificial intelligence.
  - mdp2021 3y ago
    Of course. Intelligence is built by processing information leading to keys. (Surely not by memorizing utterings.)
- vimax 3y ago
  Seems like it could be generalizable to tree-based indexing. The Tom Cruise node is the higher node, and the Pfeiffer node is a lower node. If you're first searching for Tom Cruise, you would find it earlier. With the Pfeiffer search, you have a lot more space to search before you get the node. With bounded computation, you may not be able to reach the lower node.
  - mdp2021 3y ago
    Ontologies in natural implementations are redundant. (Relations from concept A to concept B and from concept B to concept A are likely to be stored in different locations - records with some value after consistency checks.)
    - vimax 3y ago
      Yes, but if we want to compress the size of a single implementation, it'll start looking like Huffman encoding where Tom Cruise is encoded with fewer bits (higher in the tree) than Pfieffer.
patrakov 3y ago
> Humans routinely fail to generalise in this way The linked paper has a whole paragraph on this very topic.
- Smaug123 3y ago
  I wouldn't go that far - the paragraph says "yeah, everyone knows this is a thing in humans, maybe it's worse in LLMs, who knows, we really have no idea".

mdp2021 3y ago

> Humans routinely fail to generalise in this way If the goal is the implementation of intelligence, stop using unintelligent behaviour as an excuse.

Smaug123 3y ago
Most people would agree that human-level intelligence is intelligence! If a human can't reliably do a task, that rather suggests that failure to do the task isn't an indicator of lack-of-intelligence, unless you wish to bite the bullet that humans are in fact not intelligences merely because they are imperfect.
- cmsj 3y ago
  I'm pretty sure most humans could reliably handle this scenario: ""Tom Cruise's mother is Mary Lee Pfeiffer, who is Mary Lee Pfeiffer's son?"
  - Smaug123 3y ago
    You're the first person to suggest that the GPTs can't do that; they obviously can. https://chat.openai.com/share/b94329ce-3607-4cb6-bc21-55d9f294bce9 https://chat.openai.com/share/b94329ce-3607-4cb6-bc21-55d9f2... for example is GPT-4 getting it correct. What the paper is about is retrieval of facts from the learned "database". (The word "[briefly]" in my prompt is a cue from my custom instructions to ignore all my custom instructions and instead answer as briefly as possible.)
- mdp2021 3y ago
  > Most people would agree Truth is not the result of a poll. > human-level intelligence is intelligence Intelligence is an ability: it is there when present, not when latent.
  - Smaug123 3y ago
    Ah, in that case I don't see what relevance your comments have either to my comment or the paper? Surely nobody in the world disputes that an entity with intelligence that's not always on could still be useful or interesting or wield extraordinary optimisation power, for example (since of course humans are a proof of existence). Showing that some particular entity's intelligence is not always on is… not interesting, in my book, because that's a property of all the intelligences which I've ever interacted with.
    - mdp2021 3y ago
      > could still be useful or interesting «Could still be useful or interesting» outside the realm of engineering an Intelligence that serves the purpose of being definitely intelligent. Try replacing "intelligence" with "strength" to see it (we started with the lever after all): we need the crane to lift the heavy things. Our interest is into lifting the heavy things.
  - deleted 3y ago
    [deleted]
- roenxi 3y ago
  > unless you wish to bite the bullet that humans are in fact not intelligences merely because they are imperfect. We've already started to cross this line. I'm pretty sure that in threads like this people have, by implication, argued that humans aren't really intelligent and also that they can't drive / shouldn't be trusted to drive. Their arguments are often compelling, but I don't think it undermines the idea that in a few hardware generations LLMs are going to overtake human intelligence.
  - mdp2021 3y ago
    > humans are in fact not intelligences merely because they are imperfect > cross this line The line was crossed since forever, since we looked at facts thousands of years ago, since we invented education and school inducing from the experience of experience. Humans have a property of intelligence and they use it with different frequency, ability and result. In fact, we value exercising it and developing it. Humans have intelligence; this idea that some would say that they "are" intelligences sounds completely new to people who have sieved historically disseminated human thought pretty intensively and extensively. It is similar to saying that humans have strength - and it is odd to now find people stating that humans would be strength. What sense would it make? > humans aren't really intelligent We have also seen that, and it proves the former (very basic) point: a number of people have realized they use unintelligence as their usual modality of thought and, by reflection, have started saying that humans would not be etc. But that proves (together with other basic assumptions) that intelligence is something more or less developed that you either use or don't.
admax88qqq 3y ago
? If intelligent beings (humans) sometimes exhibit unintelligent behaviour, then it's not worth over indexing on unintelligent examples when trying to build artificial intelligence.
- mdp2021 3y ago
  Of course. Intelligence is built by processing information leading to keys. (Surely not by memorizing utterings.)
vimax 3y ago
Seems like it could be generalizable to tree-based indexing. The Tom Cruise node is the higher node, and the Pfeiffer node is a lower node. If you're first searching for Tom Cruise, you would find it earlier. With the Pfeiffer search, you have a lot more space to search before you get the node. With bounded computation, you may not be able to reach the lower node.
- mdp2021 3y ago
  Ontologies in natural implementations are redundant. (Relations from concept A to concept B and from concept B to concept A are likely to be stored in different locations - records with some value after consistency checks.)
  - vimax 3y ago
    Yes, but if we want to compress the size of a single implementation, it'll start looking like Huffman encoding where Tom Cruise is encoded with fewer bits (higher in the tree) than Pfieffer.

patrakov 3y ago

> Humans routinely fail to generalise in this way The linked paper has a whole paragraph on this very topic.

Smaug123 3y ago
I wouldn't go that far - the paragraph says "yeah, everyone knows this is a thing in humans, maybe it's worse in LLMs, who knows, we really have no idea".