3 ms·

I'd buy the technically impossible angle. Even if you manage to get your microservices to synch every penny spent to your payment account at realtime (impossib

I'd buy the technically impossible angle. Even if you manage to get your microservices to synch every penny spent to your payment account at realtime (impossible) you still have to waiver the excess, losing some money every time someone goes past their quota.

bartread 6mo ago
Sure, but 80 -> 28,000 -> 54,000 is a hell of a lot of slippage. Trading platforms can guarantee a maximum slippage on stops, and often even offer guaranteed stops (with an attached premium), so I don’t see why Google and Firebase can’t do similar. The way it works at present is ridiculous.
- zbentley 6mo ago
  Yep. And cloud providers could eat any slippage cost (enforcing, say, every 5 minutes by stopping service) without even a rounding error on their balance sheets. The fact that they don’t indicates that there’s no market reason to support small spenders who get mad about runaway overages, not that it’s technically or financially hard to do so.
- nurettin 6mo ago
  > Trading platforms can guarantee a maximum slippage on stops Yeah no, physically impossible. If nobody is selling at that price, there is no guarantee your sell stop will execute near that price. They can sweep the market, find the best seller price and execute. There might be a costly way to do it with microservices as I indicated, but your example easily falls apart.
  - bartread 6mo ago
    Not impossible to do: they can hedge and/or absorb the cost, hence the premium. They usually also specify a (fairly large) minimum distance for such stops.
    - nurettin 6mo ago
      That's exactly what I proposed in my response. Big corp can waiver the extra costs to match your limit. Glad we finally got to that part of my response. The question is: will they? Probably not. Do brokers do it? I haven't seen any. Maybe you know more.
  - projektfu 6mo ago
    If they are a market maker, they can buy/sell at or near your stop. It might be a bad idea for them, but if they have a guarantee, this is how they will do it. Or, it will be like the Amazon guarantee (refunding free shipping on your late order).
  - dmurray 6mo ago
    They can take the other side of your other themselves, lose money sometimes, but make it up in the premium they charged you in the first place (or in the old days, from your other trading fees or your monthly subscription payment). Cloud providers would be taking way less risk interacting with their own services than a broker does interacting with the market. Perhaps they would be more at risk from bad actors, but it shouldn't be significant: they could reserve this behaviour for people who have already spent, say, $100 with them so you can't abuse it at scale.
deleted 6mo ago
[deleted]
walthamstow 6mo ago
I'm with you. And what do you even do when the quota is breached, nuke the resources? People will complain about that just as much as overspends. I don't buy the 'evil corp screwing people' angle either. They are making farrr too much legit money to care about occasionally screwing people out of 20k and 50k.
- radlad 6mo ago
  If I set a limit, and you cut off my service because I reached the limit, I would definitely not "complain just as much" as if I set a limit and you allowed me to spend past it. We're not talking about an EC2 or EBS volume here, this is access to an API.
  - deleted 6mo ago
    [deleted]
  - walthamstow 6mo ago
    Meh, you probably would complain. Maybe you forgot you set it. Now your project is taking off, making money, and it got nuked. Why aren't we talking about an EC2 - is that not a cloud compute service? People have been complaining about cloud billing since long before LLMs. Anything to say about the technical problem of constantly monitoring many services against a project or account-level limit?
    - ChromaticPanic 6mo ago
      You mean we can implement rate limiting on API for security purposes no problem but suddenly having it track costs as well is technically impossible?
    - radlad 6mo ago
      Why aren't we talking about an EC2? Because this is a thread about the Gemini inference API. Loss of service would be restored on payment, not permanent. But that's besides the point: I as the customer set a limit, and you as as the service provider did not adhere to it. I've worked on a number of systems and while it is sometimes impossible to stop at an exact limit, I am confident that it is feasible to stop with less slippage than occurred in this scenario. And at the companies I've worked at, within a margin of error that we're able to absorb any slippage ourselves, as these losses are made up elsewhere, and are worth the customer goodwill. If we can do it, I'm sure Google can.
benterix 6mo ago
I invite you to look at the various solutions implemented by those public cloud providers that actually implemented this feature.
andreareina 6mo ago
"I can only do the job pretty damn well, not perfectly, so might as well not try."

bartread 6mo ago

Sure, but 80 -> 28,000 -> 54,000 is a hell of a lot of slippage. Trading platforms can guarantee a maximum slippage on stops, and often even offer guaranteed stops (with an attached premium), so I don’t see why Google and Firebase can’t do similar. The way it works at present is ridiculous.

zbentley 6mo ago
Yep. And cloud providers could eat any slippage cost (enforcing, say, every 5 minutes by stopping service) without even a rounding error on their balance sheets. The fact that they don’t indicates that there’s no market reason to support small spenders who get mad about runaway overages, not that it’s technically or financially hard to do so.
nurettin 6mo ago
> Trading platforms can guarantee a maximum slippage on stops Yeah no, physically impossible. If nobody is selling at that price, there is no guarantee your sell stop will execute near that price. They can sweep the market, find the best seller price and execute. There might be a costly way to do it with microservices as I indicated, but your example easily falls apart.
- bartread 6mo ago
  Not impossible to do: they can hedge and/or absorb the cost, hence the premium. They usually also specify a (fairly large) minimum distance for such stops.
  - nurettin 6mo ago
    That's exactly what I proposed in my response. Big corp can waiver the extra costs to match your limit. Glad we finally got to that part of my response. The question is: will they? Probably not. Do brokers do it? I haven't seen any. Maybe you know more.
- projektfu 6mo ago
  If they are a market maker, they can buy/sell at or near your stop. It might be a bad idea for them, but if they have a guarantee, this is how they will do it. Or, it will be like the Amazon guarantee (refunding free shipping on your late order).
- dmurray 6mo ago
  They can take the other side of your other themselves, lose money sometimes, but make it up in the premium they charged you in the first place (or in the old days, from your other trading fees or your monthly subscription payment). Cloud providers would be taking way less risk interacting with their own services than a broker does interacting with the market. Perhaps they would be more at risk from bad actors, but it shouldn't be significant: they could reserve this behaviour for people who have already spent, say, $100 with them so you can't abuse it at scale.

deleted 6mo ago

[deleted]

walthamstow 6mo ago

I'm with you. And what do you even do when the quota is breached, nuke the resources? People will complain about that just as much as overspends. I don't buy the 'evil corp screwing people' angle either. They are making farrr too much legit money to care about occasionally screwing people out of 20k and 50k.

radlad 6mo ago
If I set a limit, and you cut off my service because I reached the limit, I would definitely not "complain just as much" as if I set a limit and you allowed me to spend past it. We're not talking about an EC2 or EBS volume here, this is access to an API.
- deleted 6mo ago
  [deleted]
- walthamstow 6mo ago
  Meh, you probably would complain. Maybe you forgot you set it. Now your project is taking off, making money, and it got nuked. Why aren't we talking about an EC2 - is that not a cloud compute service? People have been complaining about cloud billing since long before LLMs. Anything to say about the technical problem of constantly monitoring many services against a project or account-level limit?
  - ChromaticPanic 6mo ago
    You mean we can implement rate limiting on API for security purposes no problem but suddenly having it track costs as well is technically impossible?
  - radlad 6mo ago
    Why aren't we talking about an EC2? Because this is a thread about the Gemini inference API. Loss of service would be restored on payment, not permanent. But that's besides the point: I as the customer set a limit, and you as as the service provider did not adhere to it. I've worked on a number of systems and while it is sometimes impossible to stop at an exact limit, I am confident that it is feasible to stop with less slippage than occurred in this scenario. And at the companies I've worked at, within a margin of error that we're able to absorb any slippage ourselves, as these losses are made up elsewhere, and are worth the customer goodwill. If we can do it, I'm sure Google can.

benterix 6mo ago

I invite you to look at the various solutions implemented by those public cloud providers that actually implemented this feature.

andreareina 6mo ago

"I can only do the job pretty damn well, not perfectly, so might as well not try."