3 ms·

I'm sorry, but I disagree. They gave the BBC enough detail that a very misleading headline was produced as a result. True, the main blame lies with the BBC, but

I'm sorry, but I disagree. They gave the BBC enough detail that a very misleading headline was produced as a result. True, the main blame lies with the BBC, but it also comes across — to me, anyway, maybe I'm being too cynical — as a bit of an excuse from Fastly.

mytailorisrich 5y ago
"But a customer quite legitimately changing their settings had exposed a bug in a software update issued to customers in mid-May, causing "85% of our network to return errors" They are careful to make clear that the customer did nothing wrong and that the problem was a bug in their software.
- oneeyedpigeon 5y ago
  I know — as I've said, the main blame lies with the bbc. However, as it's reported, it comes across very much as Fastly trying to save face. Maybe the blame is entirely on the bbc, maybe Fastly were naive in thinking that giving them this information wouldn't result in irresponsible headlines.
  - Jenk 5y ago
    How on earth do you figure that is anyone's fault but the BBC? Read Fastly's statement. There is nothing about it blaming the customer(s) at all. There is nothing trying to save face. What is your point here?
    - oneeyedpigeon 5y ago
      > Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors. Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: > Early June 8, a configuration change triggered a bug in our software, which caused 85% of our network to return errors. The advantage is that you wouldn't get ignorant reporting that "one customer took down the internet". I'm not sure there are disadvantages that net outweigh that.
      - laumars 5y ago
        > Is it necessary to refer to "a customer" at all in this statement? That’s how autopsies work. You describe the cause and resolution. The cause was a bug in the customers control panel. They’re not trying to absolve themselves of responsibility.
      - Jenk 5y ago
        Yes, because it is explaining that it was a valid *customer* configuration, which is a separate set of concerns from, say, infrastructure config. The important adjective "valid" means it was completely normal/expected input and thus not the fault of the customer. It's perfectly clear you've come at this with a pre-determined agenda of "I bet fastly, like most other public statements after corporate booboos I've seen, will try to shrug this one off as someone else's fault" after reading the BBCs title and haven't bothered to read it at all until now.
        madaxe_again 5y ago
        Not necessarily valid. Could have been a bad entry that passed validation when it shouldn’t have, which would still not be the customer’s fault.
        Jenk 5y ago
        > We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. Verbatim from Fastly: https://www.fastly.com/blog/summary-of-june-8-outage https://www.fastly.com/blog/summary-of-june-8-outage
      - perbu 5y ago
        The somewhat awkward "a customer pushed a _valid_ configuration" is Fastly making sure they aren't pushing any blame onto the customer. There is no customer blaming here. None at all.
      - staticassertion 5y ago
        > Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: That's literally what happened. They even say it was a valid configuration change, it's very blameless. Saying "a configuration change" loses critical context. I would have assumed that this was in some sort of deployment update, not something that a customer could trigger. Why would you want less information here?
  - jerf 5y ago
    What verbiage exactly are you looking for from Fastly here? I'm hearing "Nobody else did anything wrong, it was 100% a software bug on our end, and we're sorry about that." How much more responsibility are you asking them to take before you would no longer be considering them to "save face"? I'm trying to come up with an ironic exaggeration here, but I can't, because it kinda seems like Fastly has already taken 100% full responsibility and there's no room left for exaggeration.
  - bennyp101 5y ago
    Don't forget that the BBC were also initially affected by this, and jumped on it a lot sooner than most outlets, so they have skin in the game.
    - iso1210 5y ago
      Would love it if it was the BBC that triggered the problem :D

mytailorisrich 5y ago

"But a customer quite legitimately changing their settings had exposed a bug in a software update issued to customers in mid-May, causing "85% of our network to return errors" They are careful to make clear that the customer did nothing wrong and that the problem was a bug in their software.

oneeyedpigeon 5y ago
I know — as I've said, the main blame lies with the bbc. However, as it's reported, it comes across very much as Fastly trying to save face. Maybe the blame is entirely on the bbc, maybe Fastly were naive in thinking that giving them this information wouldn't result in irresponsible headlines.
- Jenk 5y ago
  How on earth do you figure that is anyone's fault but the BBC? Read Fastly's statement. There is nothing about it blaming the customer(s) at all. There is nothing trying to save face. What is your point here?
  - oneeyedpigeon 5y ago
    > Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors. Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: > Early June 8, a configuration change triggered a bug in our software, which caused 85% of our network to return errors. The advantage is that you wouldn't get ignorant reporting that "one customer took down the internet". I'm not sure there are disadvantages that net outweigh that.
    - laumars 5y ago
      > Is it necessary to refer to "a customer" at all in this statement? That’s how autopsies work. You describe the cause and resolution. The cause was a bug in the customers control panel. They’re not trying to absolve themselves of responsibility.
    - Jenk 5y ago
      Yes, because it is explaining that it was a valid *customer* configuration, which is a separate set of concerns from, say, infrastructure config. The important adjective "valid" means it was completely normal/expected input and thus not the fault of the customer. It's perfectly clear you've come at this with a pre-determined agenda of "I bet fastly, like most other public statements after corporate booboos I've seen, will try to shrug this one off as someone else's fault" after reading the BBCs title and haven't bothered to read it at all until now.
      - madaxe_again 5y ago
        Not necessarily valid. Could have been a bad entry that passed validation when it shouldn’t have, which would still not be the customer’s fault.
        Jenk 5y ago
        > We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. Verbatim from Fastly: https://www.fastly.com/blog/summary-of-june-8-outage https://www.fastly.com/blog/summary-of-june-8-outage
    - perbu 5y ago
      The somewhat awkward "a customer pushed a _valid_ configuration" is Fastly making sure they aren't pushing any blame onto the customer. There is no customer blaming here. None at all.
    - staticassertion 5y ago
      > Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: That's literally what happened. They even say it was a valid configuration change, it's very blameless. Saying "a configuration change" loses critical context. I would have assumed that this was in some sort of deployment update, not something that a customer could trigger. Why would you want less information here?
- jerf 5y ago
  What verbiage exactly are you looking for from Fastly here? I'm hearing "Nobody else did anything wrong, it was 100% a software bug on our end, and we're sorry about that." How much more responsibility are you asking them to take before you would no longer be considering them to "save face"? I'm trying to come up with an ironic exaggeration here, but I can't, because it kinda seems like Fastly has already taken 100% full responsibility and there's no room left for exaggeration.
- bennyp101 5y ago
  Don't forget that the BBC were also initially affected by this, and jumped on it a lot sooner than most outlets, so they have skin in the game.
  - iso1210 5y ago
    Would love it if it was the BBC that triggered the problem :D