3 ms·
The gap in capabilities is mostly quantitative and not qualitative.
by dgb23 18d ago
The gap in capabilities is mostly quantitative and not qualitative.
- RealityVoid 18d agoIs it? I am on the fence on this, but it does seem like there are some qualitative improvements between the models. Not related to your post, but a fact I keep mulling over. The fact I don't trust the current crop of LLM's enough and I consider LLM's as a tech will hit a ceiling pretty hard, it doesn't mean parallel improvement curves won't spring up out of other research that will lead to much higher capabilities than currently.
- zahlman 18d ago> but it does seem like there are some qualitative improvements between the models. It could easily seem that way, I think, in a "quantity has a quality of its own" kind of way. When you can come to the same conclusion faster, that lets you iterate more; and sometimes when you iterate you find more things.
- dezsiszabi 18d ago> Is it? Yes, it is.
- lionkor 18d agoMy read is that the improvements in quality are due to excessive use of "thinking" tokens (so, higher quantity and brute force), so I agree with that.