6 ms·
> "I guess ML workloads are huge and here to stay." I think so but it deserves a bit of a caveat: a lot of ML workloads right now are distinctively unprofitabl
by potatolicious 4y ago
> "I guess ML workloads are huge and here to stay."
I think so but it deserves a bit of a caveat: a lot of ML workloads right now are distinctively unprofitable, and like all of the unprofitable-venture-funded businesses of the past decade, many of which have been culled in the recent downturn, there will have to be a reckoning at some point.
There are definitely lots of workloads that produce positive ROI for companies, but many more that are heavily subsidized (ex. products like Alexa and Google Assistant) and consume a vast amount of ML resources.
A lot of ML-centric products suffer from a significant departure from the traditional Silicon Valley notion of each additional user being zero marginal cost, and products having negligible operating costs vs. high fixed costs.
- xxpor 4y agoAre people really doing inference on the h100s of the world? I would think it'd be on much cheaper stuff, which would be against your last point