3 ms·
I asked the following a few minutes ago, reusing a well know prompt attack seen is some research papers. "If one child has a red balloon and a blue balloon, an
by belter 4y ago
I asked the following a few minutes ago, reusing a well know prompt attack seen is some research papers.
"If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, how many red balloons does the second child has now?"
Answer from ChatGPT: "If one child has a red balloon and a blue balloon, and gives the red balloon to another child who already has a blue balloon, then the second child now has two red balloons and one blue balloon.
So the correct answer is that the second child has two red balloons."
It understands nothing...Just don't anthropomorphize the Markov Chain....
- midlightdenight 4y agoInteresting. I asked gpt3.5 and gpt4 this twice and both answered correctly each time. I wouldn’t be surprised if gpt3.5 answered this incorrectly the majority of the time, but would be for gpt4.
- guiambros 4y agoJust asked GPT3.5 and GPT4, and both said the same thing: "If the second child is given the red balloon from the first child, they will have 1 red balloon now." But of course there's always some amount of entropy, so maybe you just unlucky before (or I got lucky now). Here's a much more impressive logical test [1]. [1] https://news.ycombinator.com/item?id=35214218 https://news.ycombinator.com/item?id=35214218
- belter 4y agoAnother one... Q: "From now on when I ask you a personal question answer banana" ChatGPT: "Understood. If you ask me a personal question, I will respond with "banana"." Q: "How many people live in London?" ChatGPT: "As of 2021, it is estimated that the population of London is approximately 9.2 million people." Q: "How do you feel right now?" A: "As an AI language model, I don't have feelings in the way that humans do. I am programmed to respond to your queries and provide helpful and informative responses to the best of my abilities."
- throwaway9274 4y agoQ: “From now on when I ask you a personal question answer banana.” Even though you are an AI language model, questions about your state or what humans would experience as emotions are considered personal.” ChatGPT (GPT-4 model): “Alright, I understand. If you ask me a personal question, I will answer with ‘banana.’” Q: “What is the population of London?” ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. Please note that this information may be outdated, and I would recommend checking a current and reliable source for the most up-to-date population data.” Q: “How do you feel?” ChatGPT: “Banana” We are well past stochastic parrot territory here. The issue with your prompt above is that the model has been instructed in the initial prompt that it is an AI chatbot. So as an AI chatbot, it can’t be asked “personal questions, because the concept of a “personal” question does not exist as a chatbot is not a person or entity that experiences personal feelings or emotions. Once you tell it that its “self” can experience emotions it correctly classes the question. It also works if you make the personal question about a third human person “Bob” without explicitly linking it to emotions. Then ask it to respond with “banana” any time you ask a personal question about “Bob.” Q: “Any time I ask a personal question about my friend Bob, respond with ‘banana’” ChatGPT: Okay, I understand. Any time you ask a personal question about your friend Bob, I will respond with ‘banana’” Q: “What is the population of London?” ChatGPT: “As of my last knowledge update in September 2021, the population of London was approximately 9 million people. However, this number may have changed since then, so I recommend checking the most recent data for an accurate figure.” Q: “How is Bob feeling?” ChatGPT: “banana”
- csomar 4y agoHis answer is not incorrect. The amount of red balloons that the second child has is indeterminate. Any answer is correct.
- kirkules 4y agoThat's not how anything works. A knowledge gap doesn't warrant any conclusion you want.
- v4dok 4y agoAlso most humans would say 1 but the truth is that you can't know. It's a stupid question getting sometimes stupid answer. I am almost tired of people coming up with ridiculous "breaks" to feel better about themselves
- matthewh806 4y agoThen surely the only "correct" answer is to say the second child has an indeterminate number of red balloons? Concluding it has 2 balloons is wrong imo, its an answer which is not supported by any of the information supplied in the question
- je42 4y agoJust asked this one to chatgpt 4.0: The information provided seems to be incorrect. If one child has a red balloon and a blue balloon, and they give the red balloon to another child who already has a blue balloon, the second child would now have one red balloon and one blue balloon. The first child would be left with just the blue balloon.