40 ms·
There's definitely hype, but the newer models can undoubtedly do things the older ones couldn't. I had multiple long-term issues that earlier models couldn't so
by usef- 17d ago
There's definitely hype, but the newer models can undoubtedly do things the older ones couldn't. I had multiple long-term issues that earlier models couldn't solve (after repeated attempts) that fable did in 1 shot. On small models too, the differences in what I can trust them with has dramatically changed compared to a few months ago. Unless you breathlessly never touched the limitatioms of earlier models, it's very clear that the wall of limitations has been moving outward.
Note also when a new benchmark is released, older models do worse on it than recent models, despite none of them being trained to the benchmark.
- mjburgess 17d agoSure, because those businesses collect training data from users who are working on those problems. Perhaps this will never saturate and frontier users will always provide data to fill last generations gaps. My sense is the economics of that are going to collapse. It's currently extremely expensive to be on this endless retrain and inference cycle in order just to bake in additional marginal features. Maybe, maybe not. However I don't personally see anything other than 'one more leap', which might in any case arise from better integration with harnesses. I can foresee a step change due to harness reinforcement -- but other than that long mild refinements that are very expensive to acquire
- usef- 17d agoWhat tasks are you trying it with? It sounds like whatever you're doing isn't saturating the existing intelligence