4 ms·

This has absolutely nothing to do with the thundering herd problem. Why are you linking to seemingly random Wikipedia articles? The GP's question is valid: th

This has absolutely nothing to do with the thundering herd problem. Why are you linking to seemingly random Wikipedia articles? The GP's question is valid: there's no reason why server software should crash when overloaded instead of simply degrading service.

lusis 16y ago
True, there's no reason but that doesn't mean it doesn't happen. Many people (I would argue, rightly) equate degraded service to being out of service. Made-up Scenario: My cluster can handle XXXk users with an SLA of YYms response time. In degraded mode, I'm only handling XXk users with YYYYms response times. I'm not meeting my SLA for the remaining number of users so I am, in essence, offline. As to your specific point of "crashing", look at what happened with 37s. Should the server have crashed? No but there was a bug. The reason you add more capacity in the FIRST place is because the existing number of nodes cannot handle the volume. Depending on any number of bugs, issues or configuration your degraded capacity is for all intents and purposes "crashed". Made-up scenario #2: A single server in your apache configuration can handle 200k concurrent connections reliably with fast response times. Double that load and response times are so long that various devices on the path are timing out the connections as stale. Apache hasn't crashed but it's not really doing anything. Fast failure is an accepted best practice. Shit, it's baked into Erlang. Kill the process, start a new one and move on. Depending on the nature of the crash, you're doing nothing but churning processes and not actually servicing requests. The bigger problem is that people don't design for this type of scenario. Static landing pages. Decoupled services instead of monolithic all-in-one containers. Look at github. That's an awesome example of how to degrade service during an outage. Only certain components are "crashed" because everything is fairly decoupled. Meanwhile there's a guy over here running 4 apps in the same tomcat container that communicate over memory transport with each other or even if he had the common sense to decouple each app, didn't bother to fail fast and was busy spinning up threads trying to communicate with the rest of the services that he can't actually respond to anything externally.
- jemfinch 16y ago
  You're missing a fundamental difference between degraded service and being out of service: if an overloaded server handles its maximum number of clients but doesn't crash, then you have time to bring up additional servers to share the load. If an overloaded server crashes instead of limiting the number of clients it serves, you have to staunch the flow of clients further upstream before you can bring more servers up to share the load. This is what the GGGP is saying, and it seems like everyone (including you) are missing his point. Degrading service, or more accurately, serving the maximum number of clients you can serve and no more is absolutely not the same thing as being down. Failing/crashing rather than limiting the number of clients you serve is absolutely not established best practice. It's poor practice, in fact.
  - lusis 16y ago
    I'm approaching it from the business perspective. If sipport staff are getting calls for poor performance then the product isn't working. End of story. Business units don't care about shades of gray and degraded service. It's binary for them. It's either working or it isn't. Couple that with financial penalties for latency and degraded service means even less. Either you're meeting your SLA or you aren't.
moe 16y ago
This has absolutely nothing to do with the thundering herd problem. The skype outage probably not. This was in response to the thread going on about overloaded webapps - thundering herd is a common issue there. The GP's question is valid: there's no reason why server software should crash when overloaded instead of simply degrading service. Nobody used the word "crash" but graceful degradation is quite hard. You have to understand the problems at heart, plan and build for it. That is more than enough reason for most apps to not degrade very gracefully.
- jemfinch 16y ago
  Thundering herd is an efficiency problem on certain implementations of UNIX servers. It's not a common reason why servers crash when overloaded. Graceful degradation is not hard. Serve N clients, and no more. It's an extra conditional in the event loop. Am I serving N or more clients right now? Then don't accept new connections (or accept(2) them and immediately close them to avoid a backup in the listen queue).
  - moe 16y ago
    Thundering herd is an efficiency problem on certain implementations of UNIX servers. The accept()-issue you're referring to is merely a popular manifestation of the thundering herd problem. The term is nowadays also used in a broader sense to describe similar situations in distributed systems. It's not a common reason why servers crash when overloaded. Again, nobody is talking about servers crashing here. We are talking about web applications (which usually consist of multiple servers talking to each other) degrading to the point of not serving a meaningful rate of requests anymore. In that context the role of the "thundering herd" is often taken by your own users, who will furiously smash their reload-buttons to get past those 500-errors. It's an extra conditional in the event loop. Am I serving N or more clients right now? You're talking about implementation details of a socket server. I was talking about entire applications.

lusis 16y ago

True, there's no reason but that doesn't mean it doesn't happen. Many people (I would argue, rightly) equate degraded service to being out of service. Made-up Scenario: My cluster can handle XXXk users with an SLA of YYms response time. In degraded mode, I'm only handling XXk users with YYYYms response times. I'm not meeting my SLA for the remaining number of users so I am, in essence, offline. As to your specific point of "crashing", look at what happened with 37s. Should the server have crashed? No but there was a bug. The reason you add more capacity in the FIRST place is because the existing number of nodes cannot handle the volume. Depending on any number of bugs, issues or configuration your degraded capacity is for all intents and purposes "crashed". Made-up scenario #2: A single server in your apache configuration can handle 200k concurrent connections reliably with fast response times. Double that load and response times are so long that various devices on the path are timing out the connections as stale. Apache hasn't crashed but it's not really doing anything. Fast failure is an accepted best practice. Shit, it's baked into Erlang. Kill the process, start a new one and move on. Depending on the nature of the crash, you're doing nothing but churning processes and not actually servicing requests. The bigger problem is that people don't design for this type of scenario. Static landing pages. Decoupled services instead of monolithic all-in-one containers. Look at github. That's an awesome example of how to degrade service during an outage. Only certain components are "crashed" because everything is fairly decoupled. Meanwhile there's a guy over here running 4 apps in the same tomcat container that communicate over memory transport with each other or even if he had the common sense to decouple each app, didn't bother to fail fast and was busy spinning up threads trying to communicate with the rest of the services that he can't actually respond to anything externally.

jemfinch 16y ago
You're missing a fundamental difference between degraded service and being out of service: if an overloaded server handles its maximum number of clients but doesn't crash, then you have time to bring up additional servers to share the load. If an overloaded server crashes instead of limiting the number of clients it serves, you have to staunch the flow of clients further upstream before you can bring more servers up to share the load. This is what the GGGP is saying, and it seems like everyone (including you) are missing his point. Degrading service, or more accurately, serving the maximum number of clients you can serve and no more is absolutely not the same thing as being down. Failing/crashing rather than limiting the number of clients you serve is absolutely not established best practice. It's poor practice, in fact.
- lusis 16y ago
  I'm approaching it from the business perspective. If sipport staff are getting calls for poor performance then the product isn't working. End of story. Business units don't care about shades of gray and degraded service. It's binary for them. It's either working or it isn't. Couple that with financial penalties for latency and degraded service means even less. Either you're meeting your SLA or you aren't.

moe 16y ago

This has absolutely nothing to do with the thundering herd problem. The skype outage probably not. This was in response to the thread going on about overloaded webapps - thundering herd is a common issue there. The GP's question is valid: there's no reason why server software should crash when overloaded instead of simply degrading service. Nobody used the word "crash" but graceful degradation is quite hard. You have to understand the problems at heart, plan and build for it. That is more than enough reason for most apps to not degrade very gracefully.

jemfinch 16y ago
Thundering herd is an efficiency problem on certain implementations of UNIX servers. It's not a common reason why servers crash when overloaded. Graceful degradation is not hard. Serve N clients, and no more. It's an extra conditional in the event loop. Am I serving N or more clients right now? Then don't accept new connections (or accept(2) them and immediately close them to avoid a backup in the listen queue).
- moe 16y ago
  Thundering herd is an efficiency problem on certain implementations of UNIX servers. The accept()-issue you're referring to is merely a popular manifestation of the thundering herd problem. The term is nowadays also used in a broader sense to describe similar situations in distributed systems. It's not a common reason why servers crash when overloaded. Again, nobody is talking about servers crashing here. We are talking about web applications (which usually consist of multiple servers talking to each other) degrading to the point of not serving a meaningful rate of requests anymore. In that context the role of the "thundering herd" is often taken by your own users, who will furiously smash their reload-buttons to get past those 500-errors. It's an extra conditional in the event loop. Am I serving N or more clients right now? You're talking about implementation details of a socket server. I was talking about entire applications.