3 ms·
> I’m not even sure whether this is possible. Based on what's happened so far, maybe. At least that's exactly how we got to the current iteration back in 2022/
by embedding-shape 7mo ago
> I’m not even sure whether this is possible.
Based on what's happened so far, maybe. At least that's exactly how we got to the current iteration back in 2022/2023, quite literally "lets see what happens when we throw an enormous amount data at them while training" worked out up until one point, then post-training seems to have taken over where labs currently differ.
- drob518 7mo agoRight, but we played the scaling card and it worked but is now reaching limits. What is the next card? You can surely argue that we can find a new one at any time. That’s the definition of a breakthrough. I just don’t see one at the moment.
- embedding-shape 7mo ago> I just don’t see one at the moment. Did you see the one before the current one was even found? Things tend to look easy in hindsight, and borderline impossible trying to look forward. Otherwise it sounds like you're in the same spot as before :)
- drob518 7mo agoThat’s what I’m said. Breakthroughs happen. No doubt about it, and they are unpredictable. Hence a breakthrough. But right now we’re using up runway with nothing yet identified to take us to the next level. And while sometimes breakthroughs happen, sometimes they don’t.
- deleted 7mo ago[deleted]
- functional_dev 7mo agobetter tooling and integration
- davebren 7mo agoIt works as far as consolidating existing human knowledge, but general intelligence doesn't suddenly emerge from it. They're out of data now but if there was 10x more (assuming it's not rehashed from existing data) and they had a 10x larger model then it would be better but that's only because there's more that it can copy from.