4 ms·

Is there any response that would satisfy you?

by thanatos_dem 7y ago

perlgeek 7y ago
How about "we will reimburse the company for any damages if we found it was our fault"?
isoprophlex 7y ago
It's been 7 hours, some follow up answers would be nice...
- skywhopper 7y ago
  You are being unreasonable here. He promised a postmortem. I’d much rather wait a few days to get a clearly written, comprehensive analysis of the problems than to get an immediate stream of confusing and contradictory raw data. If you have ever been involved in post facto analysis of a process breakdown like this you know how hard it is to get the full picture immediately. Rushing something out does no one any favors.
- thanatos_dem 7y ago
  7 hours, on a Friday night in the headquarters time zone. This issue is resolved and is clearly not wide spread, so does getting a response on Monday or Tuesday vs right now make any difference? Companies are made of people. Let the people have a life. Their night is shitty enough as is after this, I guarantee you.
  - rat9988 7y ago
    The thing is, my business don't want to deal with people. It wants to deal with a business made of multiple people to guarantee service availability. If he cannot answer, surely someone else in DigitalOcean can?
pdimitar 7y ago
Yes. "This will not happen again, ever". People's livelihoods are at stake in DO's hosting. Canned responses and brutal account lockouts should have NEVER been on the table to begin with.
- deleted 7y ago
  [deleted]
- thanatos_dem 7y ago
  That’d be unrealistic for any company to claim, and if any company I worked with did claim that I would run for the hills. That’s akin to saying “we’ll never ship a bug”, or “we have an SLO of 100%”. That’s impossible for anyone to claim. Same goes for the response handling. There is clearly a lot of room for improvement there, but if you’re insisting on not getting canned response, that means a human needs to be involved at some point. Humans will at times be slow to respond. Humans will at times make mistakes. This is just an unavoidable reality. I get that mob mentality is strong when shit hits the fan publicly, but have a bit of empathy and think about what reasonable solutions you may come up with if you were to be in their situation, rather than asking for a “magic bullet”. I could see a good response here being an overhaul of their incident response policy, especially in terms of L1 support. Probably by beefing up the L2 staffing, and escalating issues more often and more quickly. L2 support is generally product engineers rather than dedicated support staff/contractors, so it’s more expensive to do for sure, but having engineers closer to the “front line” in responding to issues closes the loop better for integrating fixes into the product, and identifying erroneous behavior more quickly.
  - pdimitar 7y ago
    Sure, me and a lot of others react rather strongly in these situations. I agree with that but you already seem to understand the reasons. However, can you say with a straight face that the very generic message left here by DO's CTO instills confidence in you about how will they handle such situations in the future? Techies hate lawyer/corporate weasel talk. Least that person could do was do their best to speak plainly without promising the sky and the moon.
geezerjay 7y ago
> Is there any response that would satisfy you? Do you believe that a PR response made in damage control mode that actually changes nothing is something that's satisfactory? I mean, apparently this screwup was so damaging that it killed a company. What part of the PR statement addresses that precendent?

Is there any response that would satisfy you?

perlgeek 7y ago

How about "we will reimburse the company for any damages if we found it was our fault"?

isoprophlex 7y ago

It's been 7 hours, some follow up answers would be nice...

skywhopper 7y ago
You are being unreasonable here. He promised a postmortem. I’d much rather wait a few days to get a clearly written, comprehensive analysis of the problems than to get an immediate stream of confusing and contradictory raw data. If you have ever been involved in post facto analysis of a process breakdown like this you know how hard it is to get the full picture immediately. Rushing something out does no one any favors.
thanatos_dem 7y ago
7 hours, on a Friday night in the headquarters time zone. This issue is resolved and is clearly not wide spread, so does getting a response on Monday or Tuesday vs right now make any difference? Companies are made of people. Let the people have a life. Their night is shitty enough as is after this, I guarantee you.
- rat9988 7y ago
  The thing is, my business don't want to deal with people. It wants to deal with a business made of multiple people to guarantee service availability. If he cannot answer, surely someone else in DigitalOcean can?

pdimitar 7y ago

Yes. "This will not happen again, ever". People's livelihoods are at stake in DO's hosting. Canned responses and brutal account lockouts should have NEVER been on the table to begin with.

deleted 7y ago
[deleted]
thanatos_dem 7y ago
That’d be unrealistic for any company to claim, and if any company I worked with did claim that I would run for the hills. That’s akin to saying “we’ll never ship a bug”, or “we have an SLO of 100%”. That’s impossible for anyone to claim. Same goes for the response handling. There is clearly a lot of room for improvement there, but if you’re insisting on not getting canned response, that means a human needs to be involved at some point. Humans will at times be slow to respond. Humans will at times make mistakes. This is just an unavoidable reality. I get that mob mentality is strong when shit hits the fan publicly, but have a bit of empathy and think about what reasonable solutions you may come up with if you were to be in their situation, rather than asking for a “magic bullet”. I could see a good response here being an overhaul of their incident response policy, especially in terms of L1 support. Probably by beefing up the L2 staffing, and escalating issues more often and more quickly. L2 support is generally product engineers rather than dedicated support staff/contractors, so it’s more expensive to do for sure, but having engineers closer to the “front line” in responding to issues closes the loop better for integrating fixes into the product, and identifying erroneous behavior more quickly.
- pdimitar 7y ago
  Sure, me and a lot of others react rather strongly in these situations. I agree with that but you already seem to understand the reasons. However, can you say with a straight face that the very generic message left here by DO's CTO instills confidence in you about how will they handle such situations in the future? Techies hate lawyer/corporate weasel talk. Least that person could do was do their best to speak plainly without promising the sky and the moon.

geezerjay 7y ago

> Is there any response that would satisfy you? Do you believe that a PR response made in damage control mode that actually changes nothing is something that's satisfactory? I mean, apparently this screwup was so damaging that it killed a company. What part of the PR statement addresses that precendent?