3 ms·
> Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. That is somet
by roenxi 7y ago
> Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost.
That is something of an illusion.
Obviously there will be some sort of uneven distribution of computing power; some institutions will have more, some less. The institutions with more power will create models at the limit of what they can do, because that is the best use of their power.
So if the thesis of more power = more results holds then truly cutting results will always be by people with resources that are practically unattainable by everyone else. Google's AlphaGo wasn't a particularly clever model, for example. It just had a lot of horsepower behind it to train it and the various ranging shot attempts Deepmind would have gone through. Someone else would have figured it out albeit more slowly in a few years as computing power became available.
Computational power is still getting exponentially more affordable [0]. Costs aren't really rising, so much as the people who have spent more money get a few years ahead of everyone else and can preview what is about to become cheap.
[0] https://aiimpacts.org/recent-trend-in-the-cost-of-computing/ https://aiimpacts.org/recent-trend-in-the-cost-of-computing/
- lonelappde 7y agoThat article says that doubling flops/$ used to take 1 year and now takes 3. It's open question whether that time gap will confirm to grow. The recent phase shift was due to approaching an asymptote in single core performance, and switching to optimizing multicore overhead. The low hanging fruit there will be consumed as well.