2 ms·
| "what if it keeps getting better at this same rate?" All relevant and recent evidence points to logarithmic improvement, not the exponential we were told (pr
by emseetech 1y ago
| "what if it keeps getting better at this same rate?"
All relevant and recent evidence points to logarithmic improvement, not the exponential we were told (promised) in the beginning.
We're likely waiting at this point for another breakthrough on the level of the attention paper. That could be next year, it could be 5-10 years from now, it could be 50 years from now. There's no point in prediction.
- tombert 1y agoYeah, that's how I feel about it. People like to assume that progress is this steady upward line, but I think it's more like a staircase. Someone comes up with something cool, there's a lot of amazing progress in the short-to-mid term, and then things kind of level out. I mean, hell, this isn't even the first time that this has happened with AI [1]. The newer AI models are pretty cool but I think we're getting into the "leveling out" phase of it. [1] https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter
- themafia 1y agoThe main problem with the current technology, to my eye, is you need these huge multi dimensional models with extremely lossy encoding in order to implement the system on a modern CPU which is effectively a 2.5D piece of hardware that ultimately accesses a 1D array of memory. Your exponential problems have exponential problems. Scaling this system is factorially hard.
- themafia 1y ago> logarithmic improvement Relative to time. Not relative to capital investment. There it's nearly perfectly linear.
- BoiledCabbage 1y ago> All relevant and recent evidence points to logarithmic improvement, Any citations for this pretty strong assertion? And please don't reply with "oh you can just tell by feel".