4 ms·
I think DeepSeek (v3 and r1) showed us that there’s still a ton of meat on the bone for fundamental research and optimization.
by janalsncm 1y ago
I think DeepSeek (v3 and r1) showed us that there’s still a ton of meat on the bone for fundamental research and optimization.
- Lerc 1y agoAbsolutely, I have seen so many good ideas that have not yet made it into notable trained models. A lot of that is because you need to have a lot more faith than "seems like a good idea" before you spend a few million in training that depends upon it. Some of it is because when the models released now began training, a lot of those ideas hasn't been published yet. Time will resolve most of that, cheaper and more performant hardware will allow a lot of those ideas to be tested without the massive commitment required to build the leading edge models.
- Workaccount2 1y agoThe big guys are almost certainly incinerating millions a day on training "maybe it could show some promise" techniques. With the way things are right now, they are probably green lighting everything to find an edge.