5 ms·

Exactly. I'm sorry OP, but with all due respect, you're delusional if you think your strategy would improve upon AWS's availability. The only way you would do t

Exactly. I'm sorry OP, but with all due respect, you're delusional if you think your strategy would improve upon AWS's availability. The only way you would do that is to deploy a solution like you mentioned on infrastructure that has higher availability than AWS. Hint: that doesn't exist. AWS may have low statistical availability for the month, but on any longer timescale they're still the very best in the game. You need to remember that whatever caused this was a black swan event.

mnutt 10y ago
I disagree. In the last few years S3 has had multiple of these black swan events, while the reverse proxies I am talking about have pushed through hundreds of billions of responses and have had significantly fewer incidents. (In this case, none) I think the fallacy here is that you're not comparing apples to apples: I would be the last to argue that I could run a globally distributed S3 competitor better than Amazon. But I can (and have) run a massively simpler service with better overall uptime because it increases our options during upstream black swan events.
- newobj 10y ago
  "But I can (and have) run a massively simpler service with better overall uptime because it increases our options during upstream black swan events." I'm not saying it's impossible. But I am saying it's dangerous to omit from this conversation the idea that introducing the very --point of option-- can cause worse reliability than just using the downstream thing in the first place.
  - mnutt 10y ago
    Sure, any new piece of infrastructure we add has the possibility for reducing reliability. We only introduce things we think we can support, and that add value for the company. YMMV. We also have less than 15 minute incident engagement times, and don't let important pages slide through dinner. It's totally standard ops stuff: if one of the servers is down, we'll replace it when we get around to it. If they're all down, pages are going off.
mattrobenolt 10y ago
You missed the major point where we're strictly improving and not thinking we're more available. In fact, I assume that we're less available, hence why we have a fallback from our local to S3 directly since I assume our single server is going to die more often than S3.
mnutt 10y ago
Honestly, think you've taken advice that is generally true ("service providers that focus on a problem can do it better than you") and applied it as dogma even when there are plenty of cases where it is flat out incorrect.

mnutt 10y ago

I disagree. In the last few years S3 has had multiple of these black swan events, while the reverse proxies I am talking about have pushed through hundreds of billions of responses and have had significantly fewer incidents. (In this case, none) I think the fallacy here is that you're not comparing apples to apples: I would be the last to argue that I could run a globally distributed S3 competitor better than Amazon. But I can (and have) run a massively simpler service with better overall uptime because it increases our options during upstream black swan events.

newobj 10y ago
"But I can (and have) run a massively simpler service with better overall uptime because it increases our options during upstream black swan events." I'm not saying it's impossible. But I am saying it's dangerous to omit from this conversation the idea that introducing the very --point of option-- can cause worse reliability than just using the downstream thing in the first place.
- mnutt 10y ago
  Sure, any new piece of infrastructure we add has the possibility for reducing reliability. We only introduce things we think we can support, and that add value for the company. YMMV. We also have less than 15 minute incident engagement times, and don't let important pages slide through dinner. It's totally standard ops stuff: if one of the servers is down, we'll replace it when we get around to it. If they're all down, pages are going off.

mattrobenolt 10y ago

You missed the major point where we're strictly improving and not thinking we're more available. In fact, I assume that we're less available, hence why we have a fallback from our local to S3 directly since I assume our single server is going to die more often than S3.

mnutt 10y ago

Honestly, think you've taken advice that is generally true ("service providers that focus on a problem can do it better than you") and applied it as dogma even when there are plenty of cases where it is flat out incorrect.