11 ms·
Generally, all cloud providers have two ways of engaging with and billing customers. One is using a credit card, which is very easy for getting started. The oth
by intellectronica 4y ago
Generally, all cloud providers have two ways of engaging with and billing customers. One is using a credit card, which is very easy for getting started. The other one is having an established relationship with a sales / account team and setting up billing by invoice. Smaller companies sometimes don't understand that and stick with credit card billing for too long. Once you move to production and have your own customers that you can't let down, you should really make the extra effort and move to invoice billing (with all the upfront overhead that entails).
- baobabKoodaa 4y agoAh yes, it was the customer's fault. Stupid customer using method recommended by service provider.
- intellectronica 4y ago
- baobabKoodaa 4y agoFor the record, I was being sarcastic and do not actually believe it was the customer's fault. I believe this is an abhorrent way of running business by Google.
- rb666 4y agoAnd then you get hit by the car, even though the light was green. Doesn't help you, does it?
- baobabKoodaa 4y agoMore like: the driver stops and gestures me to walk over the road. As I step in front of the car, they suddenly jump on the gaspedal and drive over me.
- Aeolun 4y agoUsing Google is kinda like trying to cross a 10 lane highway without getting hit.
- intellectronica 4y agoRight, but I think your sarcasm was misplaced. It is actually possible for customers to make a mistake. These mistakes are relatively easy to prevent. It could be that in this particular case there wasn't anything the OP could have done and they are simply unlucky. But that's very unlikely.
- baobabKoodaa 4y agoI have taken your advice and will keep this in mind when dealing with cloud providers in the future. That said, I still place the blame on cloud providers when things like this happen.
- TomSwirly 4y agoAll - absolutely all - the evidence we have seems to show that Google was completely wrong - that a payment didn't go off because of a known Google outage, that the company then made a manual payment, but started to receive threatening messages, that they appealed to billing who told them they were safe, but then the site was cut off in the middle of the night on the weekend. Your argument appears to be this: "Facts are unimportant: the customer is almost certainly wrong in every case."
- intellectronica 4y agoEpistimological check-in: we don't actually have any evidence, except for a very high level story from the OP without much relevant detail. So all we have is the base rate, which is in almost all cases when there are billing problems they are at least to some extent because of negligence or fraud on the part of the customer.
- yjftsjthsd-h 4y agoThe base rate of building problems is that the customer screwed up, and the base rate of Google support problems is that Google is negligent. I'm not sure how to compare these.
- paskozdilar 4y ago
- e4325f 4y agowhoosh
- deleted 4y ago[deleted]
- helsinkiandrew 4y agoHang on - what would you do if you were running a business with hundreds of thousands of users and someone doesn't pay? Was it an accident? has the customer gone bust, or died, or are they trying to defraud you and get a month or two of free service. Credit cards are exceptionally finicky, very susceptible to fraud, and payment can often fail. They're great for one off purchases and services that don't matter if they don't go through - but not suitable unless you're going to check that payments are made, which the OP seems not to have done. Perhaps GCP notifications should be better (I've experience the similar scenario with AWS and everything got fixed within a day) - but surely its up to the customer to make sure they pay.
- nosianu 4y agoYour rhetorical questions don't match OPs story. According to OP they did pay, manually, not even by credit card. > however we went ahead and made a manual payment covering all the outstanding amount + extra (the entire last paragraph of OPs text has those and more details) I suggest not using and then arguing with and against fantasized stuff when we are talking about a concrete problem for which we have a concrete description, no? Given that OP paid, of what use is your objection about non-paying customers here? You also omit this important part: > They ... assured the project will not be suspended.
- helsinkiandrew 4y ago> Your rhetorical questions don't match OPs story. According to OP they did pay, manually, not even by credit card. That's fair enough - somehow I completely misread/understood that part of the post. But my main point was that you should never be using credit cards for services that you rely on - particularly when billing/payment is automated and hidden from you. Credit card payments and checks fail regularly (10%+ for recurring payment charges) for legitimate (you don't have funds) and less legitimate reasons (a banks AI system thinking the charge looks fraudulent due to something outside of your account etc). Vendors are likely to stop the service as soon as they can get away with so they don't loose money.
- snowwrestler 4y ago
- Hnrobert42 4y agoI don’t take GP’s comment that way. I take it as a suggestion for how to avoid issues. I try not to be quick to judge when someone is offering helpful information.
- Spooky23 4y ago
- doix 4y agoIt's like looking both ways when crossing the road despite the light telling you that you can go. Sure you have right of way and if a car hits you, it's their fault. But at the end of the day, you are the one getting hit by the car. Google is at fault for how they handled it, but the parents comment is still good advice for anyone reading this thread.
- jquery 4y agoIt sounds more like they looked both ways, they saw a car, they made eye contact and the driver gestured for them to cross, then when they began to cross the driver ran them over. And now they’re getting advice to never cross streets as a pedestrian. Sound advice can also be victim blaming.
- Nebasuke 4y agoI got hit by a car in a situation similar to this. They made eye contact (I thought), were slowing down, and still hit me anyway by accelerating, despite me having the right of way on a bike. Turns out it was the first time they used the car, 1 minute into the drive and their focus was somewhere else. I would say the advice here is, wait until the car is basically stopped before you cross the street rather than trusting it will because you think all the signs say they will. I do think this is solid advice for the production systems as well. All the signs can point to yes, but making extra assurances to avoid the bad path is valid for something critical like your production systems (or your life).
- black_puppydog 4y ago> wait until the car is basically stopped you mean like an email literally saying they acknowledge the situation and won't suspend your account?
- Nebasuke 4y agoI think an email from support is more like "confirmation that they checked whether the car has seen me". I'm not sure how successfully you've dealt with support with companies, but I have almost never been able to rely on the promises of a support agent, as they are simply not the ones actually making the decisions (in this case on whether your account gets suspended or not).
- deleted 4y ago[deleted]
- jasonlotito 4y agoIf you are providing someone a service, and you rely on a third party to provide that service, and that third party screws you over, then by extension, you are screwing over your customers. That is your fault, and your customers have every right to blame you. You are responsible for the due diligence on which providers you use. Blaming a 3rd party might absolve you of some of the blame, but your still not providing the service you agreed to provide. And before you try to suggest anything else, the condition here was "having customers you can't let down." You have customers you can't let down. That's the context.
- deleted 4y ago[deleted]
- soheil 4y agoThis is the line where HN becomes not so useful. Maybe try to imagine there are other professions beyond engineering that have existed in parallel that have their way doing things oh I don't know like following standards and best practices in accounting.
- michaelcampbell 4y agoI'll counterargue with a saying from the driving context; "The graveyard is full of people who were `right`."
- dvfjsdhgfv 4y agoI'm very happy Hetzner has all these options built in from the very start. There's no need to make any "extra effort". Actually, most of my servers are on yearly billing to make things even easier for everybody.
- junon 4y agoSomeone just the other day said Hetzner did the exact same thing though...
- dvfjsdhgfv 4y agoI'm not a blind fan of Hetzner and I'm aware of their limitations, but billing works extremely well for me, and they're very patient with me when I forget to pay an invoice (sending friendly reminders and not threatening to cut me off right away). However, I remember a case when I was the culprit - spam was being sent from one of my servers (one of the users had a common username and password, something I should have never allowed) and Hetzner notified me immediately, but they didn't block the server in any way (something I'd expect). They gave me some window for explanation/solving the problem. I was very happy with how they handled the issue - but of course others may have had different experiences.
- fabioyy 4y agoi was BANNED from hetzner when i forgot to Pay ( it has been almost 10 years ).at the time i offer to Pay any debts, but they said that i was not welcome anymore.
- oefrha 4y ago> billing works extremely well for me I suppose billing works extremely well for most GCP customers as well. As for me, I hosted a side project on Hetzner a few years back for some European presence. IIRC I used PayPal for payment and they didn't support PayPal recurring payment, so I had to settle an invoice each month. Used them for about two years without problem, then missed payment for two months during a chaotic period when I didn't notice their monthly invoice emails. I got exactly one email reminder about missed payment titled "Reminder" (yes, that's all, I got all their correspondence in front of me right now) before the final "Cancellation of Contract" email and termination of my account. At that point, politely pleading with them to reinstate the account was useless, I only got a threatening letter back demanding a wire transfer of the ~€60 I owed within three business days or risk being sent to collections. Was I in the wrong? Sure. Were they friendly or patient? No.
- klohto 4y agoWhat is this fud? Both AWS and Azure won't suspend your account for several months, even with charges failing. Especially not if you're communicating with them. It's incredibly easy to get a hold of human support in case of the other clouds.
- laegooose 4y agoyes, in my experience, AWS keeps running everything as is when there is a problem with the payment method or a disagreement about bill
- jaywalk 4y agoMy company isn't very big (we spend in the very low five-figures with Azure per month) and yet we were assigned an "Azure Success Manager" (or something close to that) who I have a standing monthly phone call with, and can reach out to at any time to oversee any issues that may come up. We actually did have a billing issue come up a while ago (it was Microsoft's fault) and although our account wasn't in any real danger of being shut off, she stayed on top of the case and made sure it got resolved quickly.
- RA_Fisher 4y agoThat’s awesome customer service! Thanks for sharing.
- htrp 4y agoThat's because Microsoft understands and hires humans for customer support.....
- wongarsu 4y ago> we spend in the very low five-figures with Azure per month That's over $100,000 a year, at that point having a Key Account Manger (or Success Manager or whatever) to talk to should be the expectation.
- jaywalk 4y agoI agree. But although I've never used GCP, I get the feeling that an account of my size would not get the same treatment.
- belter 4y agoThese type of stories, that appear regularly on HN, seem in the vast majority, related to only one specific provider.
- b112 4y agoGoogle is just horrible, when it comes to any form of support, paid or not. They make paypal look helpful by comparison. I really can't understand why anyone would use any of their services. Support counts. It's one of the first things I look at, when evaluating any external partner. What does it cost, how long does it take for it to happen, etc? Whatever anyone else says in this thread, that's what you need to determine for yourself. Whatever company, what is the support channel like, how responsive is it? If you don't know? If you can't get immediate emergency response and support, and I mean immediate, then why are you even hosting there?
- vbezhenar 4y agoGoogle tech is the best. Google invented k8s, their k8s cloud is the best in the world. There's nothing similar to GKE Autopilot. Similar things could be said about some other services. People want the best cloud and they hope that those issues won't affect them.
- unity1001 4y agoYou can just use default K8 without any added service. It works well enough. There is no obligation to use GC for K8. There are a lot of good base K8 providers.
- jhugo 4y agoWe're heavy k8s users at work. We used to use GKE, now we use a mixture of EKS and bare metal after migrating most stuff away from GCP. Honestly, GKE isn't better than k8s anywhere else in any meaningful way at all.
- andix 4y agoK8S runs well on all major cloud providers. You can run it on premise without licensing costs quite well (for example k3s, rancher, …). Or fully managed with Open Shift. AWS, Azure and GC are all on the same level. Any of them gives you „the best“ technology.
- harha 4y agoIf you’re dealing with customers that way you should have a big red warning sign everywhere to not run anything in production under any circumstances. Otherwise don’t take their money and advertise that everything will be easier and more reliable
- TomSwirly 4y ago> Smaller companies sometimes don't understand that [...] you should really make the extra effort Let me get this straight. This company does everything they were supposed to, paid on time, and Google's own error cuts them off, and you are blaming them?!?
- intellectronica 4y agoYou don't know that. Maybe that's the case (I'm sure it happens sometimes), but most likely it isn't.
- Cederfjard 4y agoI think it’s fair not to take any given story at face value, without ”receipts” to back it up. You however seem to me to go in the other direction and barely want to give OP the benefit of the doubt. What details would you like to see in order to be convinced? From your profile I clicked through to your LinkedIn via your website, and I think it would’ve been becoming of you to disclose that you were a senior engineering manager at Google Cloud up until earlier this year, given the context of the discussion and your stance on it.
- intellectronica 4y agoAnd now I work for the competition :) In any case, I know enough about how cloud services work to be able to comment intelligently, and don't have any interest in Google (except for some stock I hold indirectly via ETFs). This is all just common sense plus what I know from having worked for several cloud services providers.
- rfrey 4y agoIf you assume, without evidence, that an HN user is lying if they share a story outside your experience -- there is little reason to use HN.
- intellectronica 4y ago
- croes 4y agoAnd then an account error happens and your server goes down. I don't think OP's post is about payment options but about problem handling.
- gnfargbl 4y agoIt would be nice if Google actually told their customers that this is an option. As a result of your post, I realise that I quality for invoiced billing and have just applied for it. Thanks! If I had an "established relationship with a sales / account team" then yeah, perhaps I would have been informed. Unfortunately my experience is that unless you're spending above some relatively high threshold (>$10k/mo? >$100k/mo? >$1m/mo?) then Google don't prioritise your business.
- thedougd 4y agoCloud providers have a minimum spend for invoice billing. For GCP you need to spend at least $2500 a month for 3 months to qualify.
- XCabbage 4y agoHow would invoice billing even reduce the risk of this outcome? Surely any system with more manual and asynchronous steps involved is MORE likely to result in Google erroneously claiming you haven't paid, not less? I don't grasp what the logic behind this suggestion is even meant to be.
- intellectronica 4y agoThe invoice itself doesn't solve anything, but it requires a "handshake" step where trust is established with the payer, with the result that things don't just turn off automatically when something goes wrong.
- derefr 4y agoInvoices are indefinitely postpaid; there’s no real point at which you can really 100% declare an invoice as having “not been paid.” That’s why NET30/90/etc exist — without an incentive to pay sooner, invoices will just sit around on the receiver’s books until it’s convenient for them to pay (e.g. to make their financials for a particular quarter look good.) Invoices under contract law are like PayPal’s arbitration process: “always favours the buyer.” A seller would need an explicit pre-existing contract clause stating a due date for an invoice, in order for a due date declared on an invoice to have any force; by default, it doesn’t. And because invoice billing is something companies tend to offer mostly to companies they’re trying to stay on the good side of, they don’t tend to sit them down for a binding contract negotiation when doing so.
- patchtopic 4y agothe small azure project that got canned by Azure I was working on, credit card was the only way to pay, and Azure suspended my employers Azure account with no explanation why (though it appears in hindsight to be credit card related), but they refused to explain why. this victim blaming gaslighting is ridiculous.
- raxxorraxor 4y agoAWS seems to be patient with that. Had the wrong credit card information from a customer once. Credit card validation seems to take a while and only fails a day later or so. Put in the wrong info a few times and invoices were already piling up since I only checked once a week or so. Probably still a good tip to move to a better payment scheme.