4 ms·

I wasn't able to understand what the author was trying to say under "Connection Pools"--is he suggesting that the database should be written using node.js so th

by ericflo 15y ago

I wasn't able to understand what the author was trying to say under "Connection Pools"--is he suggesting that the database should be written using node.js so that, presumably, it can "scale out" and not require a connection pool?

superjared 15y ago
Either that, or allow for >64k simultaneous connections from node so that each of his requests has its own connection. The author doesn't seem to understand that more connections to a database does not equal higher throughput.
- kowsik 15y ago
  We had a node.js app where each inbound request would go through a set of HTTP connections to the back-end couchDB. When the concurrency reached around 1,000 then there were 1,000 requests made through 10 couchDB pipelined connections. The result was the web requests were starting to get slower and slower. Think of a 6-lane road merging into a 1-lane road. Congestion.
  - wmf 15y ago
    Sure, if you're trying to shove 1,000 things through 10 connections you're going to get contention. But you could open 10K pooled connections to several different backends and still stay under the 64K limit; this should be OK up to concurrency of ~100K. Beyond that, there's always SCTP or SPDY...
  - ericflo 15y ago
    So your assertion is that if you bumped up the CouchDB connection limit to 1,000 (believe it or not, Erlang--in addition to Node--is capable of this), then this slowdown would not have occurred?
    - kowsik 15y ago
      My point is the thing you are connecting in the back-end has to scale at the same level as node - if not better. I'm sure making independent non-pipelined requests to CouchDB will help, but if CouchDB views are slow or it can't handle the concurrency, then the web requests to node.js will start slowing down resulting in a pipeline stall.
sapphirecat 15y ago
The "speed limit" is on the outbound connections from Node to the service. Node calls connect(), and receives a new port exclusively for that one connection. A server facing the Internet can serve lots of clients because those clients have plenty of IP+port combinations to go around on their end, to allow the server to tell the difference among them even though it only has the one IP+port on its end. But 60K node.js connections from one machine, the frontend server with a single IP address, to a single IP+port on the backend server, do not have that luxury. All that identifies the connection now is the port number on the Node server, so it must be unique per connection. Connection pools attempt to mitigate the problem by inserting a manager (the pool) in the middle, to accept larger numbers of requests from Node and try to schedule them on a lower, sustainable number of connections to the backend. At least on an RDBMS, transactions require the app to have exclusive use of the connection for its request, so when all the real connections are scheduled out, new requests have to wait for an old connection to be relinquished. EDIT: Going back to the blog post, it said, "You really need your back-end services to scale out with node.js." Which I think means, your back end service should have multiple IP addresses, to alleviate the bottleneck described above.
- kowsik 15y ago
  I've added a picture to clarify which ports I'm talking about. Hopefully this clarifies things a bit.
  - sapphirecat 15y ago
    Looks pretty nice. Some pictures are worth quite a few words. Is this problem made worse by the ephemeral ports remaining unavailable after disconnect, because they're stuck in TIME_WAIT? Or does a modern TCP stack note a low RTT and release the port much sooner?
    - kowsik 15y ago
      You have to think about concurrency. If there were a total of 64K simultaneous requests to that physical instance, each of which is running 100+ apps because it's multi-tenant, this drastically reduces the number of ports available to each app. With evented IO, a socket could be open for 250 ms (db query taking time) that sucks up a port causing a potential DoS on the other apps.

by ericflo 15y ago

superjared 15y ago

Either that, or allow for >64k simultaneous connections from node so that each of his requests has its own connection. The author doesn't seem to understand that more connections to a database does not equal higher throughput.

kowsik 15y ago
We had a node.js app where each inbound request would go through a set of HTTP connections to the back-end couchDB. When the concurrency reached around 1,000 then there were 1,000 requests made through 10 couchDB pipelined connections. The result was the web requests were starting to get slower and slower. Think of a 6-lane road merging into a 1-lane road. Congestion.
- wmf 15y ago
  Sure, if you're trying to shove 1,000 things through 10 connections you're going to get contention. But you could open 10K pooled connections to several different backends and still stay under the 64K limit; this should be OK up to concurrency of ~100K. Beyond that, there's always SCTP or SPDY...
- ericflo 15y ago
  So your assertion is that if you bumped up the CouchDB connection limit to 1,000 (believe it or not, Erlang--in addition to Node--is capable of this), then this slowdown would not have occurred?
  - kowsik 15y ago
    My point is the thing you are connecting in the back-end has to scale at the same level as node - if not better. I'm sure making independent non-pipelined requests to CouchDB will help, but if CouchDB views are slow or it can't handle the concurrency, then the web requests to node.js will start slowing down resulting in a pipeline stall.

sapphirecat 15y ago

The "speed limit" is on the outbound connections from Node to the service. Node calls connect(), and receives a new port exclusively for that one connection. A server facing the Internet can serve lots of clients because those clients have plenty of IP+port combinations to go around on their end, to allow the server to tell the difference among them even though it only has the one IP+port on its end. But 60K node.js connections from one machine, the frontend server with a single IP address, to a single IP+port on the backend server, do not have that luxury. All that identifies the connection now is the port number on the Node server, so it must be unique per connection. Connection pools attempt to mitigate the problem by inserting a manager (the pool) in the middle, to accept larger numbers of requests from Node and try to schedule them on a lower, sustainable number of connections to the backend. At least on an RDBMS, transactions require the app to have exclusive use of the connection for its request, so when all the real connections are scheduled out, new requests have to wait for an old connection to be relinquished. EDIT: Going back to the blog post, it said, "You really need your back-end services to scale out with node.js." Which I think means, your back end service should have multiple IP addresses, to alleviate the bottleneck described above.

kowsik 15y ago
I've added a picture to clarify which ports I'm talking about. Hopefully this clarifies things a bit.
- sapphirecat 15y ago
  Looks pretty nice. Some pictures are worth quite a few words. Is this problem made worse by the ephemeral ports remaining unavailable after disconnect, because they're stuck in TIME_WAIT? Or does a modern TCP stack note a low RTT and release the port much sooner?
  - kowsik 15y ago
    You have to think about concurrency. If there were a total of 64K simultaneous requests to that physical instance, each of which is running 100+ apps because it's multi-tenant, this drastically reduces the number of ports available to each app. With evented IO, a socket could be open for 250 ms (db query taking time) that sucks up a port causing a potential DoS on the other apps.