14 ms·
Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time,
by seizethecheese 3y ago
Instead of the model changing, it’s equally likely that this is a cognitive illusion. A new model is initially mind-blowing and enjoys a halo effect. Over time, this fades and we become frustrated with the limitations that were there all along.
- colordrops 3y agoIt's both. OpenAI is obviously tuning the model for both computational resource constraints as well as "alignment". It's not an either-or.
- hungrigekatze 3y agoCheck out this post from a round table dialogue with Greg Brockman from OpenAI. The GPT models that were in existence / in use in early 2023 were not the performance-degraded quantized versions that are in production now: https://www.reddit.com/r/mlscaling/comments/146rgq2/chatgpt_is_running_quantized/jnro7wo/ https://www.reddit.com/r/mlscaling/comments/146rgq2/chatgpt_...
- sroussey 3y agoOh interesting. I thought that’s what turbo was.
- refulgentis 3y agoIt was, that's what the comment says?
- moffkalast 3y agoNo it's definitely changed a lot. The speedups have been massive (GPT 4 runs faster now than 3.5-turbo did at launch) and they can't be explained with just them rolling out H100s since that's just a 2x inference boost. Some unknown in-house optimization method aside, they've probably quantized the models down to a few bits of precision which increases perplexity quite a bit. They've also continued to RHLF tune to make them more in-line with their guidelines and that process has been shown to decrease overall performance before GPT 4 even launched.
- kossTKR 3y agoNo. Just to add to the many examples it was good at scandinavian languages in the beginning but now it's bad.
- andrepd 3y agoYep. It's amazing how people are taking "the reddit hivemind thinks ChatGPT was gimped" as some kind of objective fact.
- ghughes 3y agoBut given the rumored architecture (MoE) it would make complete sense for them to dynamically scale down the number of models used in the mixture during periods of peak load.
- whalesalad 3y agoIt definitely got nerfed.
- browningstreet 3y agoI've never seen "nerf" used colloquially and today i've seen it at least a half-dozen times across various sites. Y'all APIs?
- whalesalad 3y agoit's popular with gamers to describe the way certain weapons/items get modified by the game developer to perform worse. buffing is the opposite, when an item gets better.
- deleted 3y ago[deleted]
- mandmandam 3y agoI've heard nerf used colloquially since like the 90's. ?
- hospitalJail 3y agoDifferent circles. I imagine they don't game.