4 ms·

They don't exactly release this data, as you know, which is why it's so volatile - nobody has any real idea, just best guesses, and lots of different smart peop

They don't exactly release this data, as you know, which is why it's so volatile - nobody has any real idea, just best guesses, and lots of different smart people have lots of different opinions.. But i agree in practice that over time nvidia's path now depends heavily on the answer to "Or will all of the efficiency savings get immediately absorbed by the demand for better performance and feed the demand for inference?" Before, their next 5-10 years did not really depend on meaningful efficiency savings existing and getting them absorbed - nobody expected meaningful efficiency savings. Now it does depend on that

ecocentrik 2y ago
There have been continuous efficiency savings innovations for both training and inference over the last few years. Discovering and absorbing those gains is a normal part of the development process. It occurs in every industry and is facilitated in computer science by mechanisms like open source and published papers. Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments. This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.
- DannyBee 2y ago
  "There have been continuous efficiency savings innovations for both training and inference over the last few years." I'm aware - i've run the relevant teams at large scale companies like Google :) Is your claim anyone has made 98% total vs, say, 5 years ago? If so, evidence please? Otherwise, seems sort of irrelevant. AFAIK, nobody has come anywhere close to this, and the only possible way they would is through externalizing roughly all costs onto someone else. (IE certainly google TPU's are much more efficient than 5 years ago, but since you have to account for development cost, they are not saving 98% vs 5 years ago)

ecocentrik 2y ago

There have been continuous efficiency savings innovations for both training and inference over the last few years. Discovering and absorbing those gains is a normal part of the development process. It occurs in every industry and is facilitated in computer science by mechanisms like open source and published papers. Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments. This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.

DannyBee 2y ago
"There have been continuous efficiency savings innovations for both training and inference over the last few years." I'm aware - i've run the relevant teams at large scale companies like Google :) Is your claim anyone has made 98% total vs, say, 5 years ago? If so, evidence please? Otherwise, seems sort of irrelevant. AFAIK, nobody has come anywhere close to this, and the only possible way they would is through externalizing roughly all costs onto someone else. (IE certainly google TPU's are much more efficient than 5 years ago, but since you have to account for development cost, they are not saving 98% vs 5 years ago)