3 ms·
And another way to rephrase it is that almost half of the users prefer the older model, which is terrible PR.
by cm2187 2y ago
And another way to rephrase it is that almost half of the users prefer the older model, which is terrible PR.
- tgsovlerkhgsel 2y agoNot if the goal is to claim that the models deliver comparable quality, but with the new one excelling at something else (here: inferrence cost).
- swyx 2y agoits mini to mini, its the same cost
- kettleballroll 2y agoTypically in these tests you have three options "A is better", "B is better" or "they're equal/can't decide". So if 56% prefer O3 Mini, it's likely that way less than half prefer O1.also, the way I understand it, they're comparing a mini model with a large one.
- directevolve 2y agoIf you use ChatGPT, it sometimes gives you two versions of its response, and you have to choose one or the other if you want to continue prompting. Sure, not picking a response might be a third category. But if that's how they were approaching the analysis, they could have put out a more favorable-looking stat.
- ignoramous 2y ago> If you use ChatGPT, it sometimes gives you two versions Does no one else hate it when this happens (especially when on a handheld device)?