4 ms·

Does Firebase allow you to set up a billing limit, or is it one of those exciting cloud services where no matter what you do you're always one mistake away fro

Does Firebase allow you to set up a billing limit, or is it one of those exciting cloud services where no matter what you do you're always one mistake away from losing your house?

stavros 2y ago
Apparently she had a $20 limit, but, if my calculations are correct, $70k is more than that, which seems odd.
- whoisburbansky 2y ago
  Somebody else pointed out that it's likely just an alert, not a hard limit, which checks out given Firebase documentation (https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp...), which has no mention of hard limits and explicitly warns you that an alert won't stop anything.
  - stavros 2y ago
    Ah, oops...
layman51 2y ago
They have a documentation page titled “Avoid surprise bills”[1] but I imagine it’s easy for some developers to skip over that. [1]: https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp...
- unsnap_biceps 2y ago
  https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp... > We don't turn off services and usage because although you might have a bug in your app causing an increase in spend, you might just be experiencing unexpected positive growth of your app. You don't want your app to shut down unexpectedly when you need it to work the most. Frankly, I don't see anything on that page that would actually prevent a surprise bill.
- jsheard 2y ago
  > A budget alert sends an email whenever your project's spending level hits a threshold that you've set. Budget alerts do NOT turn off services or usage for your app. It's not an automatic hard-stop so you could still screw yourself over pretty badly with runaway spending.
- rezokun 2y ago
  eg. OpenAI enforces strict limits until you spend a significant amount of money.
  - mason55 2y ago
    My guess is that OpenAI’s margins are much lower so they aren’t in a position to forgive or have people skip out on big bills. For Firebase, their costs are probably pretty marginal.
  - falcor84 2y ago
    From my experience, cloud LLMs are still being used by most systems as an "additional feature" with fallback to alternative basic functionality, or some other form of graceful degradation. On the other hand, there typically isn't a good fallback to having your main DB go down.
  - jasongill 2y ago
    Once you do spend a significant amount of money, you'll find that the limits you can set yourself might not even work. We have a multi-organization OpenAI account and I had set a $4k/month limit on one of the child orgs that was being used for a R&D project. Got billed ~$20k for the project one month and complained that it clearly allowed us to exceed our soft and hard limits. We were told that the limits (which you can set and act like they are a real thing) don't do anything if you have a child organization set up. They refunded us after some persuading and I know it's just normal growing pains for an organization that is undergoing rapid growth and maturity, but was still a little surprising that even the "hard limit" didn't do anything for us! (Note that this was last year, so this bug is probably long fixed as they have redesigned their portal multiple times now)
  - danpalmer 2y ago
    OpenAI functionally have one API, it's easy to limit spend on one API. It's much less easy to limit spend across hundreds of APIs, and resources that have ongoing cost (like VM hours).
danpalmer 2y ago
I understand joking about this and the possible downsides, but there are good reasons why cloud services are set up this way: 1) it's impossible to bill at scale and exactly cut off service usage globally when a target is hit, and 2) most companies don't want a single point of failure like a misconfigured budget to bring down their production services. The answer is probably quota management, where a limit on the number of VMs or size of database or something, caps the worst-case scenario, and where it's arguably easier to monitor an approach to that quota as it's more granular than billing. Personally I think cloud providers could have an explicit "hobby mode" that limits certain things in such a way that the spend can't run away like this, with the trade-off that they're not really production grade in a sense, but then again those accounts are probably worth anything so I understand not building that out. That said, whenever I've seen one of these things happen, it always ends with "FooCloud said that as a one-time gesture of good will they would write off this accidental usage", so while briefly scary, maybe this is the system working fine overall?
- jsheard 2y ago
  Azure does actually have the ability to force-kill your resources when you hit a certain billing threshold, but only for things like free trials and student accounts. The instant you switch to a regular pay-as-you-go account that functionality disappears for uh... reasons. https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/spending-limit https://learn.microsoft.com/en-us/azure/cost-management-bill...
  - danpalmer 2y ago
    That's good, it makes sense it would be for only those sorts of accounts. Imagine building a billing system that could always do that, any time you accept a request, you need to check against a database if the user has hit their billing limit or if you can charge them for it. It obviously can't work. I imagine the way MS make it work is probably to slow down resource consumption for those accounts dramatically, and then just take the hit on overspend, knowing that it will be almost nothing.
    - 2y ago

Does Firebase allow you to set up a billing limit, or is it one of those exciting cloud services where no matter what you do you're always one mistake away from losing your house?

stavros 2y ago

Apparently she had a $20 limit, but, if my calculations are correct, $70k is more than that, which seems odd.

whoisburbansky 2y ago
Somebody else pointed out that it's likely just an alert, not a hard limit, which checks out given Firebase documentation (https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp...), which has no mention of hard limits and explicitly warns you that an alert won't stop anything.
- stavros 2y ago
  Ah, oops...

layman51 2y ago

They have a documentation page titled “Avoid surprise bills”[1] but I imagine it’s easy for some developers to skip over that. [1]: https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp...

unsnap_biceps 2y ago
https://firebase.google.com/docs/projects/billing/avoid-surprise-bills https://firebase.google.com/docs/projects/billing/avoid-surp... > We don't turn off services and usage because although you might have a bug in your app causing an increase in spend, you might just be experiencing unexpected positive growth of your app. You don't want your app to shut down unexpectedly when you need it to work the most. Frankly, I don't see anything on that page that would actually prevent a surprise bill.
jsheard 2y ago
> A budget alert sends an email whenever your project's spending level hits a threshold that you've set. Budget alerts do NOT turn off services or usage for your app. It's not an automatic hard-stop so you could still screw yourself over pretty badly with runaway spending.
rezokun 2y ago
eg. OpenAI enforces strict limits until you spend a significant amount of money.
- mason55 2y ago
  My guess is that OpenAI’s margins are much lower so they aren’t in a position to forgive or have people skip out on big bills. For Firebase, their costs are probably pretty marginal.
- falcor84 2y ago
  From my experience, cloud LLMs are still being used by most systems as an "additional feature" with fallback to alternative basic functionality, or some other form of graceful degradation. On the other hand, there typically isn't a good fallback to having your main DB go down.
- jasongill 2y ago
  Once you do spend a significant amount of money, you'll find that the limits you can set yourself might not even work. We have a multi-organization OpenAI account and I had set a $4k/month limit on one of the child orgs that was being used for a R&D project. Got billed ~$20k for the project one month and complained that it clearly allowed us to exceed our soft and hard limits. We were told that the limits (which you can set and act like they are a real thing) don't do anything if you have a child organization set up. They refunded us after some persuading and I know it's just normal growing pains for an organization that is undergoing rapid growth and maturity, but was still a little surprising that even the "hard limit" didn't do anything for us! (Note that this was last year, so this bug is probably long fixed as they have redesigned their portal multiple times now)
- danpalmer 2y ago
  OpenAI functionally have one API, it's easy to limit spend on one API. It's much less easy to limit spend across hundreds of APIs, and resources that have ongoing cost (like VM hours).

danpalmer 2y ago

I understand joking about this and the possible downsides, but there are good reasons why cloud services are set up this way: 1) it's impossible to bill at scale and exactly cut off service usage globally when a target is hit, and 2) most companies don't want a single point of failure like a misconfigured budget to bring down their production services. The answer is probably quota management, where a limit on the number of VMs or size of database or something, caps the worst-case scenario, and where it's arguably easier to monitor an approach to that quota as it's more granular than billing. Personally I think cloud providers could have an explicit "hobby mode" that limits certain things in such a way that the spend can't run away like this, with the trade-off that they're not really production grade in a sense, but then again those accounts are probably worth anything so I understand not building that out. That said, whenever I've seen one of these things happen, it always ends with "FooCloud said that as a one-time gesture of good will they would write off this accidental usage", so while briefly scary, maybe this is the system working fine overall?

jsheard 2y ago
Azure does actually have the ability to force-kill your resources when you hit a certain billing threshold, but only for things like free trials and student accounts. The instant you switch to a regular pay-as-you-go account that functionality disappears for uh... reasons. https://learn.microsoft.com/en-us/azure/cost-management-billing/manage/spending-limit https://learn.microsoft.com/en-us/azure/cost-management-bill...
- danpalmer 2y ago
  That's good, it makes sense it would be for only those sorts of accounts. Imagine building a billing system that could always do that, any time you accept a request, you need to check against a database if the user has hit their billing limit or if you can charge them for it. It obviously can't work. I imagine the way MS make it work is probably to slow down resource consumption for those accounts dramatically, and then just take the hit on overspend, knowing that it will be almost nothing.
  - 2y ago