4 ms·
> makes use of parallel test time compute Any idea what that means exactly? I vaguely remember that ChatGPT Pro was originally called "deep thought", just like
by cubefox 4mo ago
> makes use of parallel test time compute
Any idea what that means exactly? I vaguely remember that ChatGPT Pro was originally called "deep thought", just like Geminis "deep thought" feature (or "deep think"?), so it seems likely they are using the same approach.
- nl 4mo agoTheir methodology isn't published. Its widely accepted[1] that it runs the same query through the model in parallel and then has a model that either selects the best answer or synthesizes an answer from the multiple ones generated. I believe most people think it runs 6 sub-models, but I think that is based on the pricing. It's a pity that OpenAI doesn't publish details like this. [1]eg https://news.ycombinator.com/item?id=48799977 https://news.ycombinator.com/item?id=48799977
- dannyw 4mo agoBasically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks. Pro is quite limited on the web UI I reckon. This approach can be highly effective for reasonably verifiable task, for example, write comprehensive unit tests pointing out a tricky bug, get multiple agents to swarm at it.
- nl 4mo agoIt's been very successful at frontier math tasks - a bunch of the Erdos questions have been solved by it - more than any other model. https://www.erdosproblems.com/ https://www.erdosproblems.com/
- cubefox 4mo ago> Basically like passes@6 or passes@5 if you’re doing a benchmark, except for your real tasks. It's unclear how they would do this when there is no signal that provides an objective ground truth.