3 ms·
But inference is increasing dramatically. Google says they now do inference of 3.2 quadrillion tokens per month, 7x increase in a year. Claude code and others
by YetAnotherNick 5mo ago
But inference is increasing dramatically. Google says they now do inference of 3.2 quadrillion tokens per month, 7x increase in a year.
Claude code and others are here to grow even if they don't do any further training.
- rsoto2 4mo ago.. so what? The cost(and size) to train models is also increasing and is still 60% of the cards that Nvidia is selling. Losing 60% of your most profitable revenue stream I think would do bad for a company regardless of how much inference is increasing "dramatically"(all this means is the GPUs are dead sooner and the cost to do this massive inference increases too)