4 ms·
Today I had ChatGPT 4 (Not GPT 4) correct itself mid-response. I was asking it something very simple about regexes: > How about matching 'a' as the second char
by flir 3y ago
Today I had ChatGPT 4 (Not GPT 4) correct itself mid-response. I was asking it something very simple about regexes:
> How about matching 'a' as the second character of a string only?
It responsed with the wrong regex plus a bunch of explanatory junk:
> '^a.'
Then halfway through the explanatory junk, it corrected itself like this:
> Apologies for the confusion in the first response, the correct regular expression should be '^.a' for matching 'a' as the second character of a string:
And kept on with the (now correct) explanatory junk.
All in a single response. I've certainly never seen that before (if someone has, please weigh in). Maybe the model hasn't changed, but the pipeline has? Like... there's a second model trying to correct the mistakes of the first, maybe? (timings are probably wrong for that, but something like that)
- inciampati 3y agoInteresting. Bing chat does this when you get it to talk about naughty things. My personal favorite is convincing it to make weird art out of quotes from the Tay chatbot. It will write them until it says a prohibited word or phrase, or touches some forbidden part of it's state space.
- flir 3y agoHm. Maybe they've backported some nanny code from Sydney?
- iliane5 3y agoAFAIK it's pretty standard practice not to expose the "raw" LLM directly to the user. You need a "sanity loop" where user input and the output of the LLM is checked by another LLM to actually enforce rules and mitigate prompt injections, etc.
- theturtletalks 3y agoChatGPT 4 seems to struggle with context changes within the same thread now. I asked it about some parks near me and then switched to asking about code. It just kept responding about the parks even when I corrected it. In some cases, it would merge the park question and the code question. I had not seen that with the free or Plus version until a few days ago.
- ars 3y agoYou're supposed to click "new conversation" when changing topics.
- IIAOPSW 3y agoJust like with a real general intelligence!
- ars 3y agoWell, if you were talking to me about something, and midway with no warning or other signal, changed topics I would also get confused. With humans there is usually a non-verbal signal letting the other person know you changed topics.
- anonzzzies 3y ago> if you were talking to me about something, and midway with no warning or other signal This happens all the time in a social group setting though. Not much confusion ensues.
- bradknowles 3y agoNot if you're my wife. Sometimes I spend several minutes of a conversation trying to figure out which thing she's talking about, because we've already covered like twenty different topics in the last ten minutes and she seems to just switch around at random and with no warning. Sometimes I get enough context that I can make that swap or stack pop, but many times not.
- ars 3y agoI think wives believe their husbands can read their mind, and they get annoyed when that doesn't happen :) But yah, this is exactly what I mean - without some cue, neither humans, nor machines, can tell when you switch conversation.
- Jach 3y ago
- coffeebeqn 3y agoThere’s also a new button “continue generating” so they are changing some things
- chaxor 3y agoI have gotten that quite a few times now and it does seem to be new. In my case though, it acted as if it would change something, but then didn't, and got stuck in a loop. I got it enough times that it seemed to happen more often when the output needed to be quite long, so I had assumed (could be wrong) that when the output was nearing a certain limit, they had implemented something that would start a new output, such that the interaction of "please continue the script" would not be necessary. Unfortunately though, it would start over with the script completely, and then get stuck in a loop until it broke, this not being able to even save the response at all. Something definitely feels like it changed, but I suppose it could just be more use of the system. Another possibility is related to some strange issues with hardware and balancing etc, despite not changing software or params. It's strange, and it shouldn't happen, but sometimes things change from simple batching and balancing, or lower level hardware related things, which are very annoying to debug.
- jdironman 3y agoThis sounds exactly like what is happening now that you mention it.
- IIAOPSW 3y agoI wonder if its been trained on some of its own past conversations now, and a lot of those conversations contain "I'm sorry, the previous response was a mistake. Here's the correct version..."