5 ms·

Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous c

Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous connections.

Cushman 14y ago
Because that is a lot of work, and this is a drop-in solution for any node application?
lloydhilaiel 14y ago
Our problem is the the number of simultaneous connections we can support isn't static, it varies depending on the type of traffic bursts (i.e. New vs. Returning users). But agree that a higher level proxy is useful - because this way your overloaded application doesn't even need to deal with traffic (we hope to use toobusy to have applications instruct the routing layer to temporarily block / reroute traffic in times of load).
- jeffasinger 14y ago
  The reason that I asked that is I feel that most applications end up needing something at that layer eventually anyways, but not knowing exactly what level of traffic results in too much load is a good reason to do this.
  - davidw 14y ago
    It seems to me that letting the computer calculate when it's in trouble is preferable to have a human guessing.
benologist 14y ago
You'd do it because it's npm install toobusy then somewhere in your project require("toobusy"); If you only need one server that's going to be up to a lot faster than putting anything in front of it.
- byoung2 14y ago
  If you only need one server that's going to be up to a lot faster than putting anything in front of it. A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second grace), your first request will take the 5ms hit, but the next 199 for that second will be served from memory. Now with Varnish serving 199 out of 200 requests from memory, if your backend is still toobusy, by all means serve a 503, and Varnish can cache that too.
  - AndyKelley 14y ago
    You have a problem, and so you try to solve it with caching. Now you have two problems. I think that's how that quote goes.
    - byoung2 14y ago
      I think of caching for apps like clothes for people...while you could survive naked, it's more comfortable with clothes, and you're protected from heat, cold, etc. Of course there is the problem of being over/under dressed, but you have to wear something.
  - jongleberry 14y ago
    and what if your whole site is dynamic? varnish doesn't do anything.
    - byoung2 14y ago
      Very few sites are so dynamic that something can't be cached for at least a second. Even a heavily dynamic site like HN with people commenting all the time could at least cache for logged out users.
  - benologist 14y ago
    That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache. var cache; module.exports = function(request, response) { if(cache) { return response.end(cache); } // get my data from wherever cache = the_data; return response.end(cache); } Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of the language instead of a 3rd party tool you have to select and configure.
lazyjones 14y ago
I'd prefer that solution too, especially since such proxies can mitigate some types of DoS attacks (e.g. known patterns in the "Referer" header, or from known IP address ranges) and load will generally be much lower if some of the popular content is cacheable.
amatix 14y ago
Do both: each layer and service in an application should be able to manage its own load. Helps prevent cascading failures too. Read up on Netflix's architecture, where they've talked about this a lot. And the book "Release It!: Design and Deploy Production-Ready Software" (Nygard), though Java-oriented in its examples, covers the concepts well.
gfloyd 14y ago
I think this would simplify implementing a load-balancing proxy in some ways. If a server responds with a 503 generated by node-toobusy, the load balancer will know immediately to reroute the request to another server, rather than having to wait for a timeout or some other threshold.
- toong 14y ago
  So this load-balancer is balancing between A&B. Now A gives back a few 503's, you are going to send everything over to B ? [edit] I do get your point, I was just saying: don't start implementing a naive load-balancer :-)

Why you would do this, instead of putting your node server behind a more battle tested proxy, like Varnish or HAProxy, and limiting the number of simultaneous connections.

Cushman 14y ago

Because that is a lot of work, and this is a drop-in solution for any node application?

lloydhilaiel 14y ago

Our problem is the the number of simultaneous connections we can support isn't static, it varies depending on the type of traffic bursts (i.e. New vs. Returning users). But agree that a higher level proxy is useful - because this way your overloaded application doesn't even need to deal with traffic (we hope to use toobusy to have applications instruct the routing layer to temporarily block / reroute traffic in times of load).

jeffasinger 14y ago
The reason that I asked that is I feel that most applications end up needing something at that layer eventually anyways, but not knowing exactly what level of traffic results in too much load is a good reason to do this.
- davidw 14y ago
  It seems to me that letting the computer calculate when it's in trouble is preferable to have a human guessing.

benologist 14y ago

You'd do it because it's npm install toobusy then somewhere in your project require("toobusy"); If you only need one server that's going to be up to a lot faster than putting anything in front of it.

byoung2 14y ago
If you only need one server that's going to be up to a lot faster than putting anything in front of it. A single server setup might be simpler, but it won't be faster. Varnish serving a cached page from memory is going to be faster than 5 asynchronous calls that take 5ms of CPU time (and filesystem I/O in the case of the database and template given in the example). With varnish, even with a 1 second TTL (and 1 second grace), your first request will take the 5ms hit, but the next 199 for that second will be served from memory. Now with Varnish serving 199 out of 200 requests from memory, if your backend is still toobusy, by all means serve a 503, and Varnish can cache that too.
- AndyKelley 14y ago
  You have a problem, and so you try to solve it with caching. Now you have two problems. I think that's how that quote goes.
  - byoung2 14y ago
    I think of caching for apps like clothes for people...while you could survive naked, it's more comfortable with clothes, and you're protected from heat, cold, etc. Of course there is the problem of being over/under dressed, but you have to wear something.
- jongleberry 14y ago
  and what if your whole site is dynamic? varnish doesn't do anything.
  - byoung2 14y ago
    Very few sites are so dynamic that something can't be cached for at least a second. Even a heavily dynamic site like HN with people commenting all the time could at least cache for logged out users.
- benologist 14y ago
  That's super unnecessary for a single server, you can literally just use a variable outside of your request handling as a cache. var cache; module.exports = function(request, response) { if(cache) { return response.end(cache); } // get my data from wherever cache = the_data; return response.end(cache); } Now 199 out of 200 requests are from memory, there are zero extra moving parts, and you're using a cool part of the language instead of a 3rd party tool you have to select and configure.

lazyjones 14y ago

I'd prefer that solution too, especially since such proxies can mitigate some types of DoS attacks (e.g. known patterns in the "Referer" header, or from known IP address ranges) and load will generally be much lower if some of the popular content is cacheable.

amatix 14y ago

Do both: each layer and service in an application should be able to manage its own load. Helps prevent cascading failures too. Read up on Netflix's architecture, where they've talked about this a lot. And the book "Release It!: Design and Deploy Production-Ready Software" (Nygard), though Java-oriented in its examples, covers the concepts well.

gfloyd 14y ago

I think this would simplify implementing a load-balancing proxy in some ways. If a server responds with a 503 generated by node-toobusy, the load balancer will know immediately to reroute the request to another server, rather than having to wait for a timeout or some other threshold.

toong 14y ago
So this load-balancer is balancing between A&B. Now A gives back a few 503's, you are going to send everything over to B ? [edit] I do get your point, I was just saying: don't start implementing a naive load-balancer :-)