3 ms·
Meh, you probably would complain. Maybe you forgot you set it. Now your project is taking off, making money, and it got nuked. Why aren't we talking about an E
by walthamstow 6mo ago
Meh, you probably would complain. Maybe you forgot you set it. Now your project is taking off, making money, and it got nuked.
Why aren't we talking about an EC2 - is that not a cloud compute service? People have been complaining about cloud billing since long before LLMs.
Anything to say about the technical problem of constantly monitoring many services against a project or account-level limit?
- ChromaticPanic 6mo agoYou mean we can implement rate limiting on API for security purposes no problem but suddenly having it track costs as well is technically impossible?
- radlad 6mo agoWhy aren't we talking about an EC2? Because this is a thread about the Gemini inference API. Loss of service would be restored on payment, not permanent. But that's besides the point: I as the customer set a limit, and you as as the service provider did not adhere to it. I've worked on a number of systems and while it is sometimes impossible to stop at an exact limit, I am confident that it is feasible to stop with less slippage than occurred in this scenario. And at the companies I've worked at, within a margin of error that we're able to absorb any slippage ourselves, as these losses are made up elsewhere, and are worth the customer goodwill. If we can do it, I'm sure Google can.