4 ms·

But why do providers get a free pass when they do a major mistake (à la GitLab losing tons of not-backed-up data), but customers are required to accept whatever

But why do providers get a free pass when they do a major mistake (à la GitLab losing tons of not-backed-up data), but customers are required to accept whatever the provider throws at them? It seems pretty unfair to me.

moduspol 9y ago
I don't think they should get a free pass, but I don't think they should be beholden forever to be under-metering traffic if they made a mistake and weren't counting part of it (as it seems to be the case here). To that extent, they should have communicated this better, ideally 90 days out perhaps even with phone calls for customers whose bill will increase notably.
- MaulingMonkey 9y ago
  I'd even settle for crediting affected accounts to help smooth out the transition reactively, which it sounded like they started to do. But then they failed to follow through.
  - MaulingMonkey 9y ago
    They've followed through now!
MaulingMonkey 9y ago
Providers to some degree get a free pass when they do their best to fix their mistakes. In Gitlab's case, their response was excellent and their total data loss was less than a day's work - and even that only for a full time manipulator of gitlab metadata (project managers? some parts of QA?): "Database data such as projects, issues, snippets, etc. created between January 31st 17:20 UTC and 23:30 UTC has been lost. Git repositories and Wikis were not removed as they are stored separately." ( https://about.gitlab.com/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/2017/02/10/postmortem-of-database-o... ) Considering that "[...] out of 5 backup/replication techniques deployed none are working reliably or set up in the first place." ( https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCxIABGiryG7_z_6jHdVik/pub https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCx... ), this was a pretty amazing result, and about the best a response they could make. That, combined with their transparency, earns something of a pass (especially when data loss I care about is already limited by git's DVCS nature, even if they had obliterated the repository data.) In Firelab's case, if no other action is taken, they've actively mislead their customer in failing to follow through on crediting their account, charged them an obscene and unsupported price hike, and gone incommunicado. That's like... an anti-pass.
- brazzledazzle 9y ago
  Chances are they also knew when they fixed the bug that it would result in price hikes. They could have easily generated a report showing which customers would be impacted, allowing them to contact them ahead of time.
  - MaulingMonkey 9y ago
    From the followup updates, it sounds like Firebase actually tried to do this: https://medium.com/@startupandrew/firebase-founder-here-im-very-sorry-for-the-surprise-and-frustration-experienced-by-the-poster-b0065c99f53e https://medium.com/@startupandrew/firebase-founder-here-im-v... Sounds like they've processed the credits as well, so that's good.

moduspol 9y ago

I don't think they should get a free pass, but I don't think they should be beholden forever to be under-metering traffic if they made a mistake and weren't counting part of it (as it seems to be the case here). To that extent, they should have communicated this better, ideally 90 days out perhaps even with phone calls for customers whose bill will increase notably.

MaulingMonkey 9y ago
I'd even settle for crediting affected accounts to help smooth out the transition reactively, which it sounded like they started to do. But then they failed to follow through.
- MaulingMonkey 9y ago
  They've followed through now!

MaulingMonkey 9y ago

Providers to some degree get a free pass when they do their best to fix their mistakes. In Gitlab's case, their response was excellent and their total data loss was less than a day's work - and even that only for a full time manipulator of gitlab metadata (project managers? some parts of QA?): "Database data such as projects, issues, snippets, etc. created between January 31st 17:20 UTC and 23:30 UTC has been lost. Git repositories and Wikis were not removed as they are stored separately." ( https://about.gitlab.com/2017/02/10/postmortem-of-database-outage-of-january-31/ https://about.gitlab.com/2017/02/10/postmortem-of-database-o... ) Considering that "[...] out of 5 backup/replication techniques deployed none are working reliably or set up in the first place." ( https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCxIABGiryG7_z_6jHdVik/pub https://docs.google.com/document/d/1GCK53YDcBWQveod9kfzW-VCx... ), this was a pretty amazing result, and about the best a response they could make. That, combined with their transparency, earns something of a pass (especially when data loss I care about is already limited by git's DVCS nature, even if they had obliterated the repository data.) In Firelab's case, if no other action is taken, they've actively mislead their customer in failing to follow through on crediting their account, charged them an obscene and unsupported price hike, and gone incommunicado. That's like... an anti-pass.

brazzledazzle 9y ago
Chances are they also knew when they fixed the bug that it would result in price hikes. They could have easily generated a report showing which customers would be impacted, allowing them to contact them ahead of time.
- MaulingMonkey 9y ago
  From the followup updates, it sounds like Firebase actually tried to do this: https://medium.com/@startupandrew/firebase-founder-here-im-very-sorry-for-the-surprise-and-frustration-experienced-by-the-poster-b0065c99f53e https://medium.com/@startupandrew/firebase-founder-here-im-v... Sounds like they've processed the credits as well, so that's good.