4 ms·

My impression is that solar (and maybe wind?) energy have benefited from learning-by-doing [1][2] that has resulted in lower costs and/or improved performance

by slwvx 9mo ago

My impression is that solar (and maybe wind?) energy have benefited from learning-by-doing [1][2] that has resulted in lower costs and/or improved performance each year. It seems reasonable to me that a similar process will apply to AI (at least in the long run). The rate of learning could be seen as a "pace" of improvement. I'm curious, do you have a reference for the deceleration of pace that you refer to? [1] https://emp.lbl.gov/news/new-study-refocuses-learning-curve https://emp.lbl.gov/news/new-study-refocuses-learning-curve [2] https://ourworldindata.org/grapher/solar-pv-prices-vs-cumulative-capacity https://ourworldindata.org/grapher/solar-pv-prices-vs-cumula...

Jean-Papoulos 9mo ago
Why would a the curve of solar prices be in any way correlated with the curve of AI improvements ? The deceleration of pace is visible to anyone capable of using Google.
Arkhaine_kupo 9mo ago
> It seems reasonable to me that a similar process will apply to AI If its reasonable, then reason it. Because it is a highly apples to oranges comparison you are making
specialist 9mo ago
u/ipaddr is probably referring to 1) the dearth of new (novel) training data. Hence the mad scramble to hoover up, buy, steal, any potentially plausible new sources. 2) diminishing returns of embiggening compute clusters for training LLMs and size of their foundation models. (As you know) You're referring to Wright's Law aka experience learning curve. So there's a tension. Some concerns that we're nearing the ceiling for training. While the cost of applications using foundation models (implementing inference engines) is decreasing. Someone smarter than me will have to provide the slopes of the (misc) learning curves.
- slwvx 9mo ago
  I was not aware of (or had forgotten) the term "Wright's law" [1], but that indeed is what I was thinking of. It looks like some may use the term "learning curve" to refer to the same idea (efficiency gains that follow investment); the Wikipedia page on "Learning curve" [2] includes references to Wright. [1] https://en.wikipedia.org/wiki/Experience_curve_effect https://en.wikipedia.org/wiki/Experience_curve_effect [2] https://en.wikipedia.org/wiki/Learning_curve https://en.wikipedia.org/wiki/Learning_curve

by slwvx 9mo ago

Jean-Papoulos 9mo ago

Why would a the curve of solar prices be in any way correlated with the curve of AI improvements ? The deceleration of pace is visible to anyone capable of using Google.

Arkhaine_kupo 9mo ago

> It seems reasonable to me that a similar process will apply to AI If its reasonable, then reason it. Because it is a highly apples to oranges comparison you are making

specialist 9mo ago

u/ipaddr is probably referring to 1) the dearth of new (novel) training data. Hence the mad scramble to hoover up, buy, steal, any potentially plausible new sources. 2) diminishing returns of embiggening compute clusters for training LLMs and size of their foundation models. (As you know) You're referring to Wright's Law aka experience learning curve. So there's a tension. Some concerns that we're nearing the ceiling for training. While the cost of applications using foundation models (implementing inference engines) is decreasing. Someone smarter than me will have to provide the slopes of the (misc) learning curves.

slwvx 9mo ago
I was not aware of (or had forgotten) the term "Wright's law" [1], but that indeed is what I was thinking of. It looks like some may use the term "learning curve" to refer to the same idea (efficiency gains that follow investment); the Wikipedia page on "Learning curve" [2] includes references to Wright. [1] https://en.wikipedia.org/wiki/Experience_curve_effect https://en.wikipedia.org/wiki/Experience_curve_effect [2] https://en.wikipedia.org/wiki/Learning_curve https://en.wikipedia.org/wiki/Learning_curve