5 ms·

How on earth do you figure that is anyone's fault but the BBC? Read Fastly's statement. There is nothing about it blaming the customer(s) at all. There is noth

by Jenk 5y ago

How on earth do you figure that is anyone's fault but the BBC? Read Fastly's statement. There is nothing about it blaming the customer(s) at all. There is nothing trying to save face. What is your point here?

oneeyedpigeon 5y ago
> Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors. Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: > Early June 8, a configuration change triggered a bug in our software, which caused 85% of our network to return errors. The advantage is that you wouldn't get ignorant reporting that "one customer took down the internet". I'm not sure there are disadvantages that net outweigh that.
- laumars 5y ago
  > Is it necessary to refer to "a customer" at all in this statement? That’s how autopsies work. You describe the cause and resolution. The cause was a bug in the customers control panel. They’re not trying to absolve themselves of responsibility.
- Jenk 5y ago
  Yes, because it is explaining that it was a valid *customer* configuration, which is a separate set of concerns from, say, infrastructure config. The important adjective "valid" means it was completely normal/expected input and thus not the fault of the customer. It's perfectly clear you've come at this with a pre-determined agenda of "I bet fastly, like most other public statements after corporate booboos I've seen, will try to shrug this one off as someone else's fault" after reading the BBCs title and haven't bothered to read it at all until now.
  - madaxe_again 5y ago
    Not necessarily valid. Could have been a bad entry that passed validation when it shouldn’t have, which would still not be the customer’s fault.
    - Jenk 5y ago
      > We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. Verbatim from Fastly: https://www.fastly.com/blog/summary-of-june-8-outage https://www.fastly.com/blog/summary-of-june-8-outage
- perbu 5y ago
  The somewhat awkward "a customer pushed a _valid_ configuration" is Fastly making sure they aren't pushing any blame onto the customer. There is no customer blaming here. None at all.
- staticassertion 5y ago
  > Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: That's literally what happened. They even say it was a valid configuration change, it's very blameless. Saying "a configuration change" loses critical context. I would have assumed that this was in some sort of deployment update, not something that a customer could trigger. Why would you want less information here?
  - oneeyedpigeon 5y ago
    OK, I'm replying to your comment since it's the least aggressive — thanks for that! I'll fully retract my statement. This is 100% the BBC's fault, 0% Fastly's. Can I make one small suggestion that might help to prevent this kind of misleading reporting in future, though? What if Fastly produce the detailed statement they have, with as much accurate technical detail as possible AND a more general public-facing statement that organisations such as the BBC can use for reporting, that doesn't include such detailed information that can easily be misconstrued?
    - staticassertion 5y ago
      I hate being part of a dogpile, so yeah sorry about that, I just open up things to reply to, and then come back later and write it up just to find that I'm one of 10 people saying the same shit. edit: FWIW I had a very negative initial reaction to the headline as well.
      - oneeyedpigeon 5y ago
        Not at all — I understand.
    - Jenk 5y ago
      My apologies for any hostility on my part.
      - oneeyedpigeon 5y ago
- davisoneee 5y ago
  In what way is the BBC at fault for this? Their title is objectively true. A _valid_ configuration setting that was used by a customer _did_ cause fastly to have an outage. It's not limited to one specific customer (i.e this customer isn't the only customer who could have caused the issue, presumably), but it _was_ something the customer (legitimately) did. It wasn't a server outage. It wasn't a fire. It wasn't a cut cable. "a customer quite legitimately changing their settings (BBC: one fastly customer) had exposed a bug (BBC: triggered internet meltdown) in a software update issued to customers (fastly admitting, when combined with 'legitimately', that fastly are at fault) in mid-May".
  - iso1210 5y ago
    People love to hate main stream media
    - oneeyedpigeon 5y ago
      Not me — I adore the BBC. I've always paid my licence fee gladly, and I've been waxing lyrical about the latest BBC drama on Twitter just this very hour. On this issue, I believe they've made a mistake. Whatever happened to nuanced opinion, where you can see good and bad in the same entity? Why do some people insist so strongly on absolutes?
      - iso1210 5y ago
        How on earth is the headline making a mistake? Here's some excerpts: Fastly, the cloud-computing company responsible for the issues, said the bug had been triggered when one of its customers had changed their settings. Fastly senior engineering executive Nick Rockwell said: "This outage was broad and severe - and we're truly sorry for the impact to our customers and everyone who relies on them." But a customer quite legitimately changing their settings had exposed a bug in a software update issued to customers in mid-May, causing "85% of our network to return errors", it said. The headline accurately portrays the story given the limit on headlines.
- LambdaComplex 5y ago
  The wording "a configuration change triggered a bug" in this context sounds (to me) like it was a configuration change made by Fastly to something on their backend. The wording which was actually used makes it clear that that was not the case.

by Jenk 5y ago

oneeyedpigeon 5y ago

> Early June 8, a customer pushed a valid configuration change that included the specific circumstances that triggered the bug, which caused 85% of our network to return errors. Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: > Early June 8, a configuration change triggered a bug in our software, which caused 85% of our network to return errors. The advantage is that you wouldn't get ignorant reporting that "one customer took down the internet". I'm not sure there are disadvantages that net outweigh that.

laumars 5y ago
> Is it necessary to refer to "a customer" at all in this statement? That’s how autopsies work. You describe the cause and resolution. The cause was a bug in the customers control panel. They’re not trying to absolve themselves of responsibility.
Jenk 5y ago
Yes, because it is explaining that it was a valid *customer* configuration, which is a separate set of concerns from, say, infrastructure config. The important adjective "valid" means it was completely normal/expected input and thus not the fault of the customer. It's perfectly clear you've come at this with a pre-determined agenda of "I bet fastly, like most other public statements after corporate booboos I've seen, will try to shrug this one off as someone else's fault" after reading the BBCs title and haven't bothered to read it at all until now.
- madaxe_again 5y ago
  Not necessarily valid. Could have been a bad entry that passed validation when it shouldn’t have, which would still not be the customer’s fault.
  - Jenk 5y ago
    > We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. Verbatim from Fastly: https://www.fastly.com/blog/summary-of-june-8-outage https://www.fastly.com/blog/summary-of-june-8-outage
perbu 5y ago
The somewhat awkward "a customer pushed a _valid_ configuration" is Fastly making sure they aren't pushing any blame onto the customer. There is no customer blaming here. None at all.
staticassertion 5y ago
> Is it necessary to refer to "a customer" at all in this statement? What would be problematic if the above were rewritten as something like: That's literally what happened. They even say it was a valid configuration change, it's very blameless. Saying "a configuration change" loses critical context. I would have assumed that this was in some sort of deployment update, not something that a customer could trigger. Why would you want less information here?
- oneeyedpigeon 5y ago
  OK, I'm replying to your comment since it's the least aggressive — thanks for that! I'll fully retract my statement. This is 100% the BBC's fault, 0% Fastly's. Can I make one small suggestion that might help to prevent this kind of misleading reporting in future, though? What if Fastly produce the detailed statement they have, with as much accurate technical detail as possible AND a more general public-facing statement that organisations such as the BBC can use for reporting, that doesn't include such detailed information that can easily be misconstrued?
  - staticassertion 5y ago
    I hate being part of a dogpile, so yeah sorry about that, I just open up things to reply to, and then come back later and write it up just to find that I'm one of 10 people saying the same shit. edit: FWIW I had a very negative initial reaction to the headline as well.
    - oneeyedpigeon 5y ago
      Not at all — I understand.
  - Jenk 5y ago
    My apologies for any hostility on my part.
    - oneeyedpigeon 5y ago
davisoneee 5y ago
In what way is the BBC at fault for this? Their title is objectively true. A _valid_ configuration setting that was used by a customer _did_ cause fastly to have an outage. It's not limited to one specific customer (i.e this customer isn't the only customer who could have caused the issue, presumably), but it _was_ something the customer (legitimately) did. It wasn't a server outage. It wasn't a fire. It wasn't a cut cable. "a customer quite legitimately changing their settings (BBC: one fastly customer) had exposed a bug (BBC: triggered internet meltdown) in a software update issued to customers (fastly admitting, when combined with 'legitimately', that fastly are at fault) in mid-May".
- iso1210 5y ago
  People love to hate main stream media
  - oneeyedpigeon 5y ago
    Not me — I adore the BBC. I've always paid my licence fee gladly, and I've been waxing lyrical about the latest BBC drama on Twitter just this very hour. On this issue, I believe they've made a mistake. Whatever happened to nuanced opinion, where you can see good and bad in the same entity? Why do some people insist so strongly on absolutes?
    - iso1210 5y ago
      How on earth is the headline making a mistake? Here's some excerpts: Fastly, the cloud-computing company responsible for the issues, said the bug had been triggered when one of its customers had changed their settings. Fastly senior engineering executive Nick Rockwell said: "This outage was broad and severe - and we're truly sorry for the impact to our customers and everyone who relies on them." But a customer quite legitimately changing their settings had exposed a bug in a software update issued to customers in mid-May, causing "85% of our network to return errors", it said. The headline accurately portrays the story given the limit on headlines.
LambdaComplex 5y ago
The wording "a configuration change triggered a bug" in this context sounds (to me) like it was a configuration change made by Fastly to something on their backend. The wording which was actually used makes it clear that that was not the case.