3 ms·
Do we really care about this gap? If open models are 6 months to a year behind frontier models, does it really matter that much?
by LPisGood 4mo ago
Do we really care about this gap? If open models are 6 months to a year behind frontier models, does it really matter that much?
- sgc 4mo agoThis is the first time in terms of model progress where my personal response is: It does not matter to me because the models 6-12 months ago were already good enough for most everything I need to do. I think 95% of dev work is perfectly fine 6 months behind, if that is truly where we are at now with these open models.
- squidbeak 4mo agoIt depends on how far your own development tasks are beneath the higher levels of complexity and sophistication. The trade-offs for most ordinary developers will be minimal or nothing.
- LPisGood 4mo agoI believe my development tasks are some of the most complex and sophisticated out there (high performance mathematical optimization and modeling), and to me Opus 4.6 is pretty good. Which sort of development tasks do you believe are still out of reach?
- aerhardt 4mo agoI’ve seen analyses pointing to the fact that the gap is growing, which would be worrying. I think all the benchmarking and whatnot is not reliable so who knows, but we’ll definitely have a good feel in a couple of years.