3 ms·
But doesn't it feel like the larger models are seeing diminished returns compared to the smaller open models? Feels like they will eventually converge unless s
by SirMaster 2y ago
But doesn't it feel like the larger models are seeing diminished returns compared to the smaller open models?
Feels like they will eventually converge unless some big breakthrough happens that only benefits the large models.
- nyxtom 2y agoDiffusion models still have plenty of room to grow (vision/video is orders of magnitude more expensive and larger) and we're only beginning to experiment with agent (AI -> AI Agent) workflow communication and automation.
- int_19h 2y agoNot at all. There's definitely a lot of hype about how "our 7B model is almost as good as GPT-4" etc, but upon close inspection, it turns out to be cherry picked meaningless numbers every single time. At the same time, there's no evidence that even at the scale of GPT-4 they are seeing diminishing returns. I don't see why that would be the case, either, so long as there's more data to feed into the models. IMO the cliff will occur once the models become large enough that no amount of naturally sourced data is enough to saturate them, but, well, we aren't even at 1% of the Internet yet.