3 ms·
Alternatively, it’s a restatement of the obvious empirical truth that technology tends in improve on an exponential and not linear curve. Seems like a simpler e
by brookst 2y ago
Alternatively, it’s a restatement of the obvious empirical truth that technology tends in improve on an exponential and not linear curve. Seems like a simpler explanation that doesn’t even require insulting people.
- Yossarrian22 2y agoThe premise would be better supported if it could be shown that if we could 10x the speed at which matrix multiplication is performed conferred a linear or better increase in performance post GPT-4. As it stands that would just seem to give us current results faster, not better results
- brookst 2y agoEfficiency matters but it took semiconductors decades to care about it. Why would it be different this time around?
- deleted 2y ago[deleted]
- suddenlybananas 2y agoThat's why airplanes are the so much faster than they were 20 years ago.
- einsteinx2 2y agoI would argue that any given technology tends to improve on an S curve, so exponentially at first and then flattening out. See Moore’s law as a great example. Or more on topic see the improvements in LLMs since they were invented. At first each release was an order of magnitude better than the last (see GPT 2 vs 3 vs 4), now they’re getting better but at a much slower rate. Certainly feels like being at the top of an S curve to me, at least until an entirely new architecture is invented to supersede transformers.