3 ms·
Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next
by sk11001 2y ago
Recently I'm getting some strange behavior from ChatGPT (with GPT-4) where it outputs a code snippet in which it declares a variable, and then on the very next line it forgets the name of the variable and refers to it as something else. Or it refers to the variable but it makes a typo in the name.
If that's the behavior by one of the best models on a few lines of code, I'm not hopeful for a world simulator any time soon unless we see another big leap in model quality.
- therobots927 2y agoI feel like it’s an open secret at this point that chatGPT is not useful for anything requiring consistent, logical thought which is a requirement for most human jobs.
- nodja 2y agoChat GPT4 has been in general both better and worse than 3.5. I sometimes start a conversation with GPT4 and then have 3.5 fix it in another window (by pasting the code and pretending it's mine). It feels like things have degraded for coding tasks specifically, but the more recent knowledge of GPT4 is helpful if I'm asking how to use a certain library that's more recent as it won't just pretend the library exists, GPT4 is also much slower. I wish there was a ChatGPT 3.5 updated with more recent knowledge.
- catlifeonmars 2y agoThink about what the model is doing: It’s sampling from a statistical distribution of likely tokens, one of which is the correctly spelled variable name.
- sk11001 2y agoMy point is that in previous versions it was sampling more correctly more often than it has been recently.
- rany_ 2y agoFrom what I've heard, the reason it's doing that is to add an AI fingerprint. It has to pick less likely tokens to encode this information to the detriment of the output quality. Unfortunately it's only hearsay but it made sense to me so I thought I'd share.
- elliotto 2y agoPeople seem to have noticed that chatgpt has good days and bad days. we don't really know what openai is doing in the background, so it could be that some days you get assigned a really bad B in an A/B model deployment test, or maybe they're throttling the performance because they're at peak, or likely some other factor. But it's an observed phenomenon that it performs well and then doesn't.
- TeMPOraL 2y agoYes. The presence of the correctly spelled name earlier in the context should've dominated the distribution so much that it should've been extremely unlikely to select an incorrectly spelled variable name in the place where the correct name should go.
- catlifeonmars 2y agoWhat if the model got more training data? There’s no guarantee that the training data causes the model to converge on correctness. It’s not like this stuff is curated for specific problems.
- salamo 2y agoYes, I've noticed it this week as well. Simple errors like misspelling variable names.