5 ms·
I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure. Is ther
by it_citizen 3y ago
I keep reading “GPT4 got nerfed” but I have been using from day 1, and while it definitely gives bad answers, I cannot say that it was nerfed for sure.
Is there any actual evidences other than some user subjective experiences?
- dr-detroit 3y ago[dead]
- mike_hearn 3y agoChatGPT is definitely more restricted than the API. Example: https://news.ycombinator.com/item?id=36179783 https://news.ycombinator.com/item?id=36179783
- azemetre 3y agoThat's disappointing, I thought ChatGPT WAS using the API. I mean what's the point of paying if you don't get similar levels of quality?
- mike_hearn 3y agoI thought that too. It's certainly how they present it. But, apparently not.
- fredoliveira 3y agoChatGPT doesn't use the API. It uses the same underlying model with a bunch of added prompts (and possibly additional fine-tuning?) to add to make it conversational. One would pay because what they get out of chatGPT provides value, of course. Keep in mind that the users of these 2 products can be (and in fact are) different — chatGPT is a lot friendlier (from a UX perspective) than using the API playground (or using the API itself).
- redox99 3y agoThey are comparing text-davinci-003 with ChatGPT which presumably uses gpt-3.5-turbo, so quite different models. They are killing text-davinci-003 btw.
- pram 3y agoI've spent like $600 on text-davinci-003. This sucks!
- mike_hearn 3y agoWe also compare ChatGPT4 vs GPT4 API in that thread and observe the same difference.
- londons_explore 3y agoI think the clearest evidence is Microsofts paper where they show abilities at various stages during training[1]... But in a talk [2], they give more details... The unicorn gets worse during the finetuning process. [2]: https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s https://www.youtube.com/watch?v=qbIk7-JPB2c&t=1392s [1]: https://arxiv.org/abs/2303.12712 https://arxiv.org/abs/2303.12712
- it_citizen 3y agoThanks, that’s interesting. Noobie follow up question: Should we put any trust into “Sparks of intelligence” I thought it was regarded as a Microsoft marketing piece, not a serious paper.
- londons_explore 3y agoThe data presented is true... The text might be rather exaggerated/unscientific/marketing... Also notable that the team behind that paper wasn't involved in designing/building the model, but they did get access to prerelease versions.
- ChatGTP 3y agoI don’t trust it because enough third parties were able to verify the findings. This is the double edge sword of being so ridiculously closed.
- hungrigekatze 3y agoSee my comment elsewhere on this post. Greg Brockman, head of strategic initiatives at OpenAI, was talking at a round table discussion in Korea a few weeks ago about how they had to start using the quantized (smaller, cheaper) model earlier in 2023. I noticed a switch in March 2023, with GPT-4 performance being severely degraded after that for both English-language tasks as well as code-related tasks (reading and writing).
- yard2010 3y agoOh my god, this is how a lemon market[0] starts.. [0] https://en.m.wikipedia.org/wiki/The_Market_for_Lemons https://en.m.wikipedia.org/wiki/The_Market_for_Lemons