3 ms·

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them

>Product Manager for Cloud Firestore here. It's worth noting we do have the ability to set hard daily caps, as well as budgets that can have alerts tied to them. It's something we also looking at ways to improve it. Google Cloud user here. A warning: If you ever happen to get, say, frontpage on reddit or techcrunch or other big boost to publicity, your site could be down until the next billing cycle (i.e. 24 hours) and you will have no way to fix it. This bit me hard one day with appengine and lost us a ton of converting traffic, even though we tried to get the limit increased within ten minutes of the spike (and well before our limit was hit).

iamgopal 8y ago
So basically we want to have cake and eat it too.
- ajmurmann 8y ago
  Or have it be adjustable within a reasonable time frame?
- paranoidrobot 8y ago
  This seems like an overly cynical/snarky response. It's not an unreasonable request that for services which advertise the ability to scale up and down on demand, that the billing and billing limits should also be able to respond similarly.
  - badlucklottery 8y ago
    >that the billing and billing limits should also be able to respond similarly How so? With a pay-as-you-go system, firing off warnings and giving a projection of their future costs (which is hard when startups tend to have spikey traffic) is about as good as you can do. Edit: I should add that the common solution to controlling your billing in situations like this is having some overflow path built into your beta app ("Sorry, we're not taking new users at the moment" or the like).
    - Dylan16807 8y ago
      The problem is that the user gets those alerts, tries to change the limit they set, and it doesn't work. That is definitely not as good as can be done.
itcmcgrath 8y ago
Urgh, that's terrible! That definitely shouldn't be the case. I'd love for you to send me any support case you had so I can review. Even if the front door of the system didn't help you, we definitely shouldn't have been able to get you in a good state much quicker. My profile has Twitter and my DMs are open (I can give you my email there too). Doing my kids dinner, so responses might be slightly delayed.
- cheeze 8y ago
  This is exactly why I can't trust Google services though. The services are rock solid, but far far too often do I see folks who have to reach out to a dev on Twitter or their friend at Google to get a simple billing error resolved correctly. Its a problem with most cloud providers, but Google seems to be notorious for it.
  - welanes 8y ago
    From my experience Firebase are different. They're now a part of Google but support has been excellent anytime I needed it - both before and after they were acquired.
  - deadbunny 8y ago
    In my experience none of those folks are paying for support.
  - Humdeee 8y ago
    It's quite the phenomenon. Reaching out via social media to garner a company's attention when all other avenues have been exhausted... what a world we live in. "Company ignoring you? Send out a tweet, that'll work!" And it blows my mind that it actually does. It's very sad.
benguild 8y ago
Yeah, that would happen with the Google Maps API before. 150k request limit was not enough for a front-paging, and there was no rollover.
onion2k 8y ago
Your app shouldn't just fail if an API is unavailable. You should code things to fail gracefully. In the case of a traffic spike replacing the front end with an email capture form and a message saying "things are really popular right now, how about we remind you about this tomorrow (with a discount code!)" works well. This is hard for some things, but your startup failing because you didn't want to do it is much harder in the end.
- asadjb 8y ago
  Most applications have atleast 1 SPOF, the DB. Just because the DB is now the Firebase API doesn't change anything. It would be very difficult to build products in a reasonable amount of time if everything has to be coded defensively. I can build my app quicker, and remain sane, if I assume that the DB will always be available, and just fail if the DB isn't there. Same for things like S3 (which I think had only 1 large scale failure in recent history), Redis, etc... There are APIs which can be unavailable and you need to work around those. For me, these are mostly third party services that I don't control. But then again, I'm not building the next Netflix. I don't have enough engineers to build an app that works with a chaos monkey! Not the best approach for all applications, but has been good enough for most projects I've worked on. Just my 2 cents.
stickfigure 8y ago
This is a pretty common train of thought, but it's not really a question of deciding between "my bill will explode!" and "my app will go down!" You just really have to put some serious thought into what your daily limits should be, and add some reasonable alerting to detect surges. The tools are there and they're not terribly hard to use. It just tends to be an afterthought for most developers because this doesn't look like a customer feature. Without these sorts of automated scaled services, the traditional behavior is "your app goes down". This is a big improvement!
- jliptzin 8y ago
  No, if your app is failing in a way that a page view now costs $1.00 instead of $0.0001 for example then the application should go down.

iamgopal 8y ago

So basically we want to have cake and eat it too.

ajmurmann 8y ago
Or have it be adjustable within a reasonable time frame?
paranoidrobot 8y ago
This seems like an overly cynical/snarky response. It's not an unreasonable request that for services which advertise the ability to scale up and down on demand, that the billing and billing limits should also be able to respond similarly.
- badlucklottery 8y ago
  >that the billing and billing limits should also be able to respond similarly How so? With a pay-as-you-go system, firing off warnings and giving a projection of their future costs (which is hard when startups tend to have spikey traffic) is about as good as you can do. Edit: I should add that the common solution to controlling your billing in situations like this is having some overflow path built into your beta app ("Sorry, we're not taking new users at the moment" or the like).
  - Dylan16807 8y ago
    The problem is that the user gets those alerts, tries to change the limit they set, and it doesn't work. That is definitely not as good as can be done.

itcmcgrath 8y ago

Urgh, that's terrible! That definitely shouldn't be the case. I'd love for you to send me any support case you had so I can review. Even if the front door of the system didn't help you, we definitely shouldn't have been able to get you in a good state much quicker. My profile has Twitter and my DMs are open (I can give you my email there too). Doing my kids dinner, so responses might be slightly delayed.

cheeze 8y ago
This is exactly why I can't trust Google services though. The services are rock solid, but far far too often do I see folks who have to reach out to a dev on Twitter or their friend at Google to get a simple billing error resolved correctly. Its a problem with most cloud providers, but Google seems to be notorious for it.
- welanes 8y ago
  From my experience Firebase are different. They're now a part of Google but support has been excellent anytime I needed it - both before and after they were acquired.
- deadbunny 8y ago
  In my experience none of those folks are paying for support.
- Humdeee 8y ago
  It's quite the phenomenon. Reaching out via social media to garner a company's attention when all other avenues have been exhausted... what a world we live in. "Company ignoring you? Send out a tweet, that'll work!" And it blows my mind that it actually does. It's very sad.

benguild 8y ago

Yeah, that would happen with the Google Maps API before. 150k request limit was not enough for a front-paging, and there was no rollover.

onion2k 8y ago

Your app shouldn't just fail if an API is unavailable. You should code things to fail gracefully. In the case of a traffic spike replacing the front end with an email capture form and a message saying "things are really popular right now, how about we remind you about this tomorrow (with a discount code!)" works well. This is hard for some things, but your startup failing because you didn't want to do it is much harder in the end.

asadjb 8y ago
Most applications have atleast 1 SPOF, the DB. Just because the DB is now the Firebase API doesn't change anything. It would be very difficult to build products in a reasonable amount of time if everything has to be coded defensively. I can build my app quicker, and remain sane, if I assume that the DB will always be available, and just fail if the DB isn't there. Same for things like S3 (which I think had only 1 large scale failure in recent history), Redis, etc... There are APIs which can be unavailable and you need to work around those. For me, these are mostly third party services that I don't control. But then again, I'm not building the next Netflix. I don't have enough engineers to build an app that works with a chaos monkey! Not the best approach for all applications, but has been good enough for most projects I've worked on. Just my 2 cents.

stickfigure 8y ago

This is a pretty common train of thought, but it's not really a question of deciding between "my bill will explode!" and "my app will go down!" You just really have to put some serious thought into what your daily limits should be, and add some reasonable alerting to detect surges. The tools are there and they're not terribly hard to use. It just tends to be an afterthought for most developers because this doesn't look like a customer feature. Without these sorts of automated scaled services, the traditional behavior is "your app goes down". This is a big improvement!

jliptzin 8y ago
No, if your app is failing in a way that a page view now costs $1.00 instead of $0.0001 for example then the application should go down.