4 ms·

Look at christkv comment for "why this is not a good idea in node.js". You may be thinking you can fire up multiple request at the same time (ie: fire the requ

Look at christkv comment for "why this is not a good idea in node.js". You may be thinking you can fire up multiple request at the same time (ie: fire the requests, let the I/O do the job, get all your requests back) but since you are in a single-threaded environment, you do exactly the same as I/O on a single connection (plus the overhead of opening new sockets and establishing hands shaking with the server).

andypants 14y ago
christkv's comment is about creating a new connection per request. As I said, I agree, you should use a connection pool instead. When the web request comes in, you already have multiple database connections waiting for you. The connection overhead is not an issue then. And yes, that is exactly what I am thinking. More database connections = more concurrent database requests. I don't understand your reasoning about being in a single threaded environment. You made a statement but you haven't explained it. From what I can see, this problem is simply serial vs parallel. Is there something I'm missing? I don't see how the fact that node.js is single threaded changes the situation since the database requests aren't working within the node.js context. Edit: sorry, my previous reply may have been confusing. I was responding to your comment that having multiple connections is useless in a single-threaded environment. I was not suggesting that it may be helpful for the OP to make a new connection for each incoming http request.
- joshguthrie 14y ago
  Why not try to benchmark the results and see how it really goes? :-) In both cases, the same process of reading/writing to one or more connection sockets happen, you send and receive the same content to MongoDb: you can not send twice as much queries, you're gonna send a query, THEN you're gonna send another, then you get an answer THEN you get another answer. Everything happens in the same sequence. Also take in account MongoDb's own concurrency properties.
  - andypants 14y ago
    Well then why would the nodejs mongodb driver support connection pooling?
    - christkv 14y ago
      for a very simple reason :) the current mongodb database has a thread pinned to each socket connection. This thread will serially execute the incoming wire protocol messages. So say you have 2 queries running at the same time. The first query takes 230 ms and the second one takes 100 ms. If you use only a single socket the first one will have to finish processing before the second one can be processed. This means that the total time of execution is 330 ms. If you fire each message on it's own socket they will be executed in parallel. The balance is between how big your pool is and how many messages are in flight. In the future this will change as MongoDB moves to some sort of eventloop or multiplexing of threads to sockets. But for the moment you have to strike a balance between operations in flight and how many sockets you open on the server. Another thing to keep in mind is that the more socket connections you open on the mongodb server the more memory needs to allocated on the heap and the more contexts switches between threads will happen.

andypants 14y ago

christkv's comment is about creating a new connection per request. As I said, I agree, you should use a connection pool instead. When the web request comes in, you already have multiple database connections waiting for you. The connection overhead is not an issue then. And yes, that is exactly what I am thinking. More database connections = more concurrent database requests. I don't understand your reasoning about being in a single threaded environment. You made a statement but you haven't explained it. From what I can see, this problem is simply serial vs parallel. Is there something I'm missing? I don't see how the fact that node.js is single threaded changes the situation since the database requests aren't working within the node.js context. Edit: sorry, my previous reply may have been confusing. I was responding to your comment that having multiple connections is useless in a single-threaded environment. I was not suggesting that it may be helpful for the OP to make a new connection for each incoming http request.

joshguthrie 14y ago
Why not try to benchmark the results and see how it really goes? :-) In both cases, the same process of reading/writing to one or more connection sockets happen, you send and receive the same content to MongoDb: you can not send twice as much queries, you're gonna send a query, THEN you're gonna send another, then you get an answer THEN you get another answer. Everything happens in the same sequence. Also take in account MongoDb's own concurrency properties.
- andypants 14y ago
  Well then why would the nodejs mongodb driver support connection pooling?
  - christkv 14y ago
    for a very simple reason :) the current mongodb database has a thread pinned to each socket connection. This thread will serially execute the incoming wire protocol messages. So say you have 2 queries running at the same time. The first query takes 230 ms and the second one takes 100 ms. If you use only a single socket the first one will have to finish processing before the second one can be processed. This means that the total time of execution is 330 ms. If you fire each message on it's own socket they will be executed in parallel. The balance is between how big your pool is and how many messages are in flight. In the future this will change as MongoDB moves to some sort of eventloop or multiplexing of threads to sockets. But for the moment you have to strike a balance between operations in flight and how many sockets you open on the server. Another thing to keep in mind is that the more socket connections you open on the mongodb server the more memory needs to allocated on the heap and the more contexts switches between threads will happen.