5 ms·
The author is assuming inference will eventually get broken across many cheaper machines. To their credit, if an nvidia box is $50,000 and a cpu box is $5,000
by ip26 3y ago
The author is assuming inference will eventually get broken across many cheaper machines.
To their credit, if an nvidia box is $50,000 and a cpu box is $5,000 then a cluster of ten cpu boxes is only one order of magnitude behind.
- YetAnotherNick 3y agoI think that(no source but heard from few folks) if they run at full capacity, electricity cost will get larger than the base cost in few years. And energy per flop is an order of magnitude lower in GPU.
- deleted 3y ago[deleted]