4 ms·

Your post isn't providing any argument, you're just providing statement after statement of a position that is pretty well known (no emergent behavior, just patt

Your post isn't providing any argument, you're just providing statement after statement of a position that is pretty well known (no emergent behavior, just pattern matching) and pretty popular here on hackernews. Maybe the link provides all the good stuff, but I'd suggest you at least provide some flavor of what your arguments are.

fsndz 2y ago
Each argument could easily fill a book. Some valuable insights are discussed in the linked article, but most of the evidence comes from a series of research papers demonstrating that LLMs perform poorly when confronted with out-of-training distribution prompts. For example, even state-of-the-art LLMs can fail simple tasks like counting the number of "r" letters in the word "strawberry" or fall victim to the reversal curse (e.g., LLMs trained on "A is B" often struggle to deduce "B is A"). Another interesting read on this topic comes from MIT's research, which further explores these limitations: (https://news.mit.edu/2024/reasoning-skills-large-language-models-often-overestimated-0711 https://news.mit.edu/2024/reasoning-skills-large-language-mo...)
- tbrownaw 2y ago
  > or fall victim to the reversal curse (e.g., LLMs trained on "A is B" often struggle to deduce "B is A"). But should "Joe is human" logically imply that "human is Joe"?
  - fsndz 2y ago
    The reversal curse is mostly about understanding specific logical relationships rather than identity statements. It's not about statements like 'Joe is human' implying 'human is Joe,' but rather about LLMs struggling to reverse relationships or roles, such as turning 'A is B' into 'B is A.' Check out the summary of the paper: "We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Valentina Tereshkova was the first woman to travel to space", it will not automatically be able to answer the question, "Who was the first woman to travel to space?". Moreover, the likelihood of the correct answer ("Valentina Tershkova") will not be higher than for a random name. Thus, models do not generalize a prevalent pattern in their training set: if "A is B" occurs, "B is A" is more likely to occur. It is worth noting, however, that if "A is B" appears in-context, models can deduce the reverse relationship. We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. Code available at: this https URL."

fsndz 2y ago

Each argument could easily fill a book. Some valuable insights are discussed in the linked article, but most of the evidence comes from a series of research papers demonstrating that LLMs perform poorly when confronted with out-of-training distribution prompts. For example, even state-of-the-art LLMs can fail simple tasks like counting the number of "r" letters in the word "strawberry" or fall victim to the reversal curse (e.g., LLMs trained on "A is B" often struggle to deduce "B is A"). Another interesting read on this topic comes from MIT's research, which further explores these limitations: (https://news.mit.edu/2024/reasoning-skills-large-language-models-often-overestimated-0711 https://news.mit.edu/2024/reasoning-skills-large-language-mo...)

tbrownaw 2y ago
> or fall victim to the reversal curse (e.g., LLMs trained on "A is B" often struggle to deduce "B is A"). But should "Joe is human" logically imply that "human is Joe"?
- fsndz 2y ago
  The reversal curse is mostly about understanding specific logical relationships rather than identity statements. It's not about statements like 'Joe is human' implying 'human is Joe,' but rather about LLMs struggling to reverse relationships or roles, such as turning 'A is B' into 'B is A.' Check out the summary of the paper: "We expose a surprising failure of generalization in auto-regressive large language models (LLMs). If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. For instance, if a model is trained on "Valentina Tereshkova was the first woman to travel to space", it will not automatically be able to answer the question, "Who was the first woman to travel to space?". Moreover, the likelihood of the correct answer ("Valentina Tershkova") will not be higher than for a random name. Thus, models do not generalize a prevalent pattern in their training set: if "A is B" occurs, "B is A" is more likely to occur. It is worth noting, however, that if "A is B" appears in-context, models can deduce the reverse relationship. We provide evidence for the Reversal Curse by finetuning GPT-3 and Llama-1 on fictitious statements such as "Uriah Hawthorne is the composer of Abyssal Melodies" and showing that they fail to correctly answer "Who composed Abyssal Melodies?". The Reversal Curse is robust across model sizes and model families and is not alleviated by data augmentation. We also evaluate ChatGPT (GPT-3.5 and GPT-4) on questions about real-world celebrities, such as "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?". GPT-4 correctly answers questions like the former 79% of the time, compared to 33% for the latter. Code available at: this https URL."