I agree with most of the criticisms here, and will add on one more: while it is generally true that you can’t beat “serverless” inference pricing for LLMs, prod

I agree with most of the criticisms here, and will add on one more: while it is generally true that you can’t beat “serverless” inference pricing for LLMs, production deployments often depend on fine-tuned models, for which these providers typically charge much more to host. That’s where the cost (and security, etc.) advantage for running on dedicated hardware comes in.