4 ms·
I think what it comes down to is accuracy vs speed. OpenAI clearly took steps here to improve the accuracy of the output which is critical in a lot of cases for
by trash_cat 2y ago
I think what it comes down to is accuracy vs speed. OpenAI clearly took steps here to improve the accuracy of the output which is critical in a lot of cases for application. Even if it will take longer, I think this is a good direction. I am a bit skeptical when it comes to the benchmarks - because they can be gamed and they don't always reflect real world scenarios. Let's see how it works when people get to apply it in real life workflows. One last thing, I wish they could elaborate more on >>"We have found that the performance of o1 consistently improves with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute)."<< Why don't you keep training it for years then to approach 100%? Am I missing something here?
- zamadatix 2y agoThose scales are log so "years" more training may be an improvement but in absolute terms it may not be worth running. One point at which the cost vs return doesn't make sense to keep running, another point at which new approaches to building LLMs can quickly give a better result than training the old model's training for years would give anyways. There is probably also a practical limit at which it does truly flatten, it's probably just well past either of those points so it might as well not exist.