2 ms·
I used to run a lot of monte carlo simulations where the error is proportional to the inverse square root. There was a huge advantage of running for an hour vs
by norir 2y ago
I used to run a lot of monte carlo simulations where the error is proportional to the inverse square root. There was a huge advantage of running for an hour vs a few minutes, but you hit the diminishing returns depressingly quickly. It would not surprise me at all if llms end up having similar scaling properties.
- riku_iki 2y agoAnd I suspect o3 is something like monte carlo: generates tons of CoTs, with most of them are junk, but some hit the answer.
- exhaze 2y agoSounds plausible given I’ve recently observed a ton of research papers in the space that in some way or another incorporate MCTS
- LegionMammal978 2y agoYeah, any situation you need O(n^2) runtime to obtain n bits of output (or bits of accuracy, in the Monre Carlo case) is pure pain. At every point, it's still within your means to double the amount of output (by running it 3x longer than you have so far), but it gradually becomes more and more painful, instead of there being a single point where you can call it off.