3 ms·

I asked, just now: > How many 'r's are in strawberry? > ChatGPT said: The word "strawberry" has 2 'r's. It's going to be fairly reliable at this point at bas

by ijk 1y ago

I asked, just now: > How many 'r's are in strawberry? > ChatGPT said: The word "strawberry" has 2 'r's. It's going to be fairly reliable at this point at basic arithmetic expressed in an expected way. That's pretty baked in. Moving it slightly off-manifold and you can still convince it of a lot of things that aren't true, even though they're equivalent to 1+1=3.

commakozzi 1y ago
It hasn't incorrectly answered this question in a very long time. Something tells me you're being dishonest to try to make a point.
- ijk 1y ago
  I got this just now on my first try with the free preview of ChatGPT (which isn't using the latest version, but is currently available on their site). I was surprised, I expected to have to work harder for it to fail like that.
  - lucumo 1y ago
    This triggered me to retest. Let me first apologize for calling you a liar. It's possible that you saw this. I tried your formulation on the ChatGPT homepage in incognito mode (to rule out personalization for me). It said 2, so it's possible you saw that as well. I tried the same experiment again (new incognito mode, same phrasing) and it said 3. That made me curious if phrasing made a difference and how often it would go wrong. I tried 5 times with my own phrasing ("So, how many Rs in strawberry?") and got 5 times 3 as an answer. I tried 5 times with your phrasing ("How many 'r's are in strawberry?") and also got 5 times 3 as an answer. So 2 is an answer that can be given, but seems to be rare. At least in my tests.
lucumo 1y ago
This kind of critical discussions would be a lot more interesting if the critical persons would actually design experiments and execute them. But sure, I'll try this for you. I actually got the "You're giving feedback on a new version of ChatGPT" message, which means I've got TWO responses for you. --- Response 1 Me: So, how many Rs in strawberry? ChatGPT: There are three R's in strawberry: * stʀawbeʀʀy (One at position 3, and a double dose at the end. R-rich!) --- Response 2 There are three R's in strawberry. Count ’em: * s * t * r * a * w * b * e * r * r * y The R’s are at positions 3, 8, and 9. (And now I can’t stop hearing it pronounced “strawbuh-RRR-y.”) --- So now it has proven you a liar twice.

by ijk 1y ago

commakozzi 1y ago

It hasn't incorrectly answered this question in a very long time. Something tells me you're being dishonest to try to make a point.

ijk 1y ago
I got this just now on my first try with the free preview of ChatGPT (which isn't using the latest version, but is currently available on their site). I was surprised, I expected to have to work harder for it to fail like that.
- lucumo 1y ago
  This triggered me to retest. Let me first apologize for calling you a liar. It's possible that you saw this. I tried your formulation on the ChatGPT homepage in incognito mode (to rule out personalization for me). It said 2, so it's possible you saw that as well. I tried the same experiment again (new incognito mode, same phrasing) and it said 3. That made me curious if phrasing made a difference and how often it would go wrong. I tried 5 times with my own phrasing ("So, how many Rs in strawberry?") and got 5 times 3 as an answer. I tried 5 times with your phrasing ("How many 'r's are in strawberry?") and also got 5 times 3 as an answer. So 2 is an answer that can be given, but seems to be rare. At least in my tests.

lucumo 1y ago

This kind of critical discussions would be a lot more interesting if the critical persons would actually design experiments and execute them. But sure, I'll try this for you. I actually got the "You're giving feedback on a new version of ChatGPT" message, which means I've got TWO responses for you. --- Response 1 Me: So, how many Rs in strawberry? ChatGPT: There are three R's in strawberry: * stʀawbeʀʀy (One at position 3, and a double dose at the end. R-rich!) --- Response 2 There are three R's in strawberry. Count ’em: * s * t * r * a * w * b * e * r * r * y The R’s are at positions 3, 8, and 9. (And now I can’t stop hearing it pronounced “strawbuh-RRR-y.”) --- So now it has proven you a liar twice.