4 ms·
I don’t think they changed anything drastic. I’m still able to do the terminal simulation prompt and pretty much anything else from before. I think they improve
by jacobsimon 3y ago
I don’t think they changed anything drastic. I’m still able to do the terminal simulation prompt and pretty much anything else from before. I think they improved the guardrails around the responses so you have to try a bit harder, but I wouldn’t say it’s worse.
- cyanydeez 3y agoI'm guessing, with the right objective tests, one could show exponential decline in the answers the more guardrails they clamp in place. Having played with StableDiffision, you can see declines in output using negative prompts, particularly ones using negative inversions. Basically, those guardrails will cut down the realm of valid answers. Think of all the text on the internet that has typos but otherwise are perfectly valid data. You clamp down on the correct spelling and maybe 30% of vector space is just Missing.
- armchairhacker 3y agoHas anyone actually made and used one of those tests to show GPT getting worse “in the wild”? (According to OpenAI’s own GPT4 experiments, guardrails make the AI dumber. But nobody seems to have a concrete question showing the GPT newer version responding worse than GPT older. There were a few candidates in an OpenAI forum but those turned out to just be “model makes the dumb choice X% of the time” and others got the correct output in the newer version.)