3 ms·
Isn't inference cheap? Why are AI labs charging so much for it?
by maplethorpe 4mo ago
Isn't inference cheap? Why are AI labs charging so much for it?
- slashdave 4mo agoDefine "cheap". If you mean cheaper than training, sure
- maplethorpe 4mo ago"Inference is cheap" is a common refrain on here. I was hoping one of the people who regularly spouts this phrase would pipe up and tell me why it's suddenly so expensive. Also, to answer your question, I think a good baseline for "cheap" is probably whether you can charge the end user enough to make a profit and not have them complain about the price of your service, as they're currently doing. If you can't do that, you're probably not even in the ballpark of "cheap", so it's not worth getting any more specific.