3 ms·
I just don't take any of the testing of these seriously anymore, OG GPT4 before the "GPT Store" and "Dev Day" this time last year still just felt way more power
by whywhywhywhy 2y ago
I just don't take any of the testing of these seriously anymore, OG GPT4 before the "GPT Store" and "Dev Day" this time last year still just felt way more powerful than anything on the market today.
3.5 Sonnet felt close but even that feels like it's being downgraded now.
Just feels like things are silently being quantized or downgraded in other ways to increase profitability once people jump on. Even trying same prompts that one shotted flawlessly pre-Dev Day 2023 now just fail for me on GPT.
- Wheatman 2y agoThis seems to be a prevalent problem that people bring up. I remember there was a forum post about it that got a lot of flack, but maybe they were onto something. Here it is: https://community.openai.com/t/did-chatgpt-4o-get-progressively-dumber-for-anyone-else-lately/846498 https://community.openai.com/t/did-chatgpt-4o-get-progressiv...
- Der_Einzige 2y agoI’m going to laugh so hard if we finally find out that quantization really does hurt models, and all the claims of 99.9999% at 4bit or lower precision really were full of it! Of all the gaslighting I see in Ml/AI, the idea that quantization is almost free is up there. I don’t buy it for one second at all, no matter how many charts you try to show me implying that the logprobs are 99% the same. Sure, maybe until you go to 10K tokens context window!
- Jensson 2y agoIts probably kind of like dementia, 99% of what they say is like before they had early dementia but its still a big problem.