3 ms·

There have been continuous efficiency savings innovations for both training and inference over the last few years. Discovering and absorbing those gains is a no

There have been continuous efficiency savings innovations for both training and inference over the last few years. Discovering and absorbing those gains is a normal part of the development process. It occurs in every industry and is facilitated in computer science by mechanisms like open source and published papers. Those savings don't usually make national news because they aren't the target. They aren't "man on the moon" moments, they are "our cannonball flies 96% as far as yours using 30% less powder" moments. This is still a race for tools that essentially brute force their way to greater utility. Until we hit a utility quotient or hit a wall, the race doesn't really end. The quickest and easiest path to advantage in the race is still more compute.

DannyBee 2y ago
"There have been continuous efficiency savings innovations for both training and inference over the last few years." I'm aware - i've run the relevant teams at large scale companies like Google :) Is your claim anyone has made 98% total vs, say, 5 years ago? If so, evidence please? Otherwise, seems sort of irrelevant. AFAIK, nobody has come anywhere close to this, and the only possible way they would is through externalizing roughly all costs onto someone else. (IE certainly google TPU's are much more efficient than 5 years ago, but since you have to account for development cost, they are not saving 98% vs 5 years ago)