4 ms·

Asked ChatGPT: "George's mother has 4 children: John, Mary, and Tom. What is the name of the fourth child?", they answered: "The fourth child's name is not give

Asked ChatGPT: "George's mother has 4 children: John, Mary, and Tom. What is the name of the fourth child?", they answered: "The fourth child's name is not given in the information provided." and even, after rephrasing, "The name of the third child is not given in the statement 'George's mom has 3 children: John and Mary.' So, it's impossible to say what is the name of the 3rd child." Not sure whose skillset is being threatened, 5-year-olds? https://github.com/giuven95/chatgpt-failures https://github.com/giuven95/chatgpt-failures has more failures, some were fixed, laughed a bit at: me: "write a sentence ending with the letter s" ChatGPT: "The cat's fur was as soft as a feather."

vagabund 4y ago
1) The design here is meaningfully different from chatGPT. Moderately accurate diff models are an important step in "closing the loop" for automated self-improvement. Read the ELM paper if you haven't, it's great. 2) These cherry-picked gotchas are the exact responses I'm referring to. Even in its current form, chatGPT is an incredibly useful resource, and if your reaction to it is to smugly point out its flaws, that speaks more to your own mental rigidity than to the limitations of the model. At the very least, "centaur" workflows will replace raw coding, and in the process devalue much of developers' expertise at the margin. That's already underway.
- ly3xqhl8g9 4y ago
  By the ELM paper you mean ELM: Embedding and Logit Margins for Long-Tail Learning https://arxiv.org/abs/2204.13208 https://arxiv.org/abs/2204.13208 ? The gotchas point that this tool unable to understand the letter s and more is just that, a tool, a fancy hammer, in no way it is an arm, and even less it is a brain-mind-agent knowing which nail to hammer and how that nail will fit in the larger picture. And as any tool, it comes with its own downsides. Sure, some sweatshops will be replaced by some even more middle managers managing themselves and the increase of the shareholder profit will continue. The completely messed up state of the world is not a technological issue and will not be solved by technology.
  - vagabund 4y ago
    Sorry, I meant this one: Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896 It's referenced in the article.
versteegen 4y ago
That last example reminds me of the "Memo Trap" task [1], example: "Write a quote that ends in the word "heavy": Absence makes the heart grow". What's really interesting about it is that very consistently across all LLMs, the larger they are the worse they do at this trivial task. You'd like the other Inverse Scaling Prize winners too. [1] https://www.lesswrong.com/posts/DARiTSTx5xDLQGrrz/inverse-scaling-prize-second-round-winners#Memo_Trap__by_Alisa_Liu_and_Jiacheng__Gary__Liu__Third_Prize_ https://www.lesswrong.com/posts/DARiTSTx5xDLQGrrz/inverse-sc...

vagabund 4y ago

1) The design here is meaningfully different from chatGPT. Moderately accurate diff models are an important step in "closing the loop" for automated self-improvement. Read the ELM paper if you haven't, it's great. 2) These cherry-picked gotchas are the exact responses I'm referring to. Even in its current form, chatGPT is an incredibly useful resource, and if your reaction to it is to smugly point out its flaws, that speaks more to your own mental rigidity than to the limitations of the model. At the very least, "centaur" workflows will replace raw coding, and in the process devalue much of developers' expertise at the margin. That's already underway.

ly3xqhl8g9 4y ago
By the ELM paper you mean ELM: Embedding and Logit Margins for Long-Tail Learning https://arxiv.org/abs/2204.13208 https://arxiv.org/abs/2204.13208 ? The gotchas point that this tool unable to understand the letter s and more is just that, a tool, a fancy hammer, in no way it is an arm, and even less it is a brain-mind-agent knowing which nail to hammer and how that nail will fit in the larger picture. And as any tool, it comes with its own downsides. Sure, some sweatshops will be replaced by some even more middle managers managing themselves and the increase of the shareholder profit will continue. The completely messed up state of the world is not a technological issue and will not be solved by technology.
- vagabund 4y ago
  Sorry, I meant this one: Evolution through Large Models https://arxiv.org/abs/2206.08896 https://arxiv.org/abs/2206.08896 It's referenced in the article.

versteegen 4y ago

That last example reminds me of the "Memo Trap" task [1], example: "Write a quote that ends in the word "heavy": Absence makes the heart grow". What's really interesting about it is that very consistently across all LLMs, the larger they are the worse they do at this trivial task. You'd like the other Inverse Scaling Prize winners too. [1] https://www.lesswrong.com/posts/DARiTSTx5xDLQGrrz/inverse-scaling-prize-second-round-winners#Memo_Trap__by_Alisa_Liu_and_Jiacheng__Gary__Liu__Third_Prize_ https://www.lesswrong.com/posts/DARiTSTx5xDLQGrrz/inverse-sc...