2 ms·

Coming from WAMP background, I had the same question when I first tried MongoDb databases with Node.JS and the consensus I read online back then (so, 3 years ag

Coming from WAMP background, I had the same question when I first tried MongoDb databases with Node.JS and the consensus I read online back then (so, 3 years ago) was "Just connect once and don't bother anymore". Though the overhead can be rather minimal, it still exists: - Connect to the database - Query the Collection - Make a request - Close the connection Also, these four separate request are asynchronous, which means you could end up with X web requests having connected to the DB but not having a collection, Y requests having a collection and N requests waiting to be closed. This is not a real problem (as long as you don't have 4 servers connecting to the same Db with 65 thousand connections each :-) ) but having multiple CONCURRENT connections to the same server is pretty useless in a SINGLE-THREADED context like a Node.js app. Now, if you were to use PHP, that's a different matter...

andypants 14y ago
Actually, it can help in a node.js app if a single db connection can only complete one request at a time. The javascript in node.js executes in a single thread but a database connection is I/O. This is exactly what node.js is good at. With that said, you shouldn't create a connection per request, you should use a connection pool.
- joshguthrie 14y ago
  Look at christkv comment for "why this is not a good idea in node.js". You may be thinking you can fire up multiple request at the same time (ie: fire the requests, let the I/O do the job, get all your requests back) but since you are in a single-threaded environment, you do exactly the same as I/O on a single connection (plus the overhead of opening new sockets and establishing hands shaking with the server).
  - andypants 14y ago
    christkv's comment is about creating a new connection per request. As I said, I agree, you should use a connection pool instead. When the web request comes in, you already have multiple database connections waiting for you. The connection overhead is not an issue then. And yes, that is exactly what I am thinking. More database connections = more concurrent database requests. I don't understand your reasoning about being in a single threaded environment. You made a statement but you haven't explained it. From what I can see, this problem is simply serial vs parallel. Is there something I'm missing? I don't see how the fact that node.js is single threaded changes the situation since the database requests aren't working within the node.js context. Edit: sorry, my previous reply may have been confusing. I was responding to your comment that having multiple connections is useless in a single-threaded environment. I was not suggesting that it may be helpful for the OP to make a new connection for each incoming http request.
    - joshguthrie 14y ago
      Why not try to benchmark the results and see how it really goes? :-) In both cases, the same process of reading/writing to one or more connection sockets happen, you send and receive the same content to MongoDb: you can not send twice as much queries, you're gonna send a query, THEN you're gonna send another, then you get an answer THEN you get another answer. Everything happens in the same sequence. Also take in account MongoDb's own concurrency properties.
      - andypants 14y ago
        Well then why would the nodejs mongodb driver support connection pooling?
        christkv 14y ago
        for a very simple reason :) the current mongodb database has a thread pinned to each socket connection. This thread will serially execute the incoming wire protocol messages. So say you have 2 queries running at the same time. The first query takes 230 ms and the second one takes 100 ms. If you use only a single socket the first one will have to finish processing before the second one can be processed. This means that the total time of execution is 330 ms. If you fire each message on it's own socket they will be executed in parallel. The balance is between how big your pool is and how many messages are in flight. In the future this will change as MongoDB moves to some sort of eventloop or multiplexing of threads to sockets. But for the moment you have to strike a balance between operations in flight and how many sockets you open on the server. Another thing to keep in mind is that the more socket connections you open on the mongodb server the more memory needs to allocated on the heap and the more contexts switches between threads will happen.

andypants 14y ago

Actually, it can help in a node.js app if a single db connection can only complete one request at a time. The javascript in node.js executes in a single thread but a database connection is I/O. This is exactly what node.js is good at. With that said, you shouldn't create a connection per request, you should use a connection pool.

joshguthrie 14y ago
Look at christkv comment for "why this is not a good idea in node.js". You may be thinking you can fire up multiple request at the same time (ie: fire the requests, let the I/O do the job, get all your requests back) but since you are in a single-threaded environment, you do exactly the same as I/O on a single connection (plus the overhead of opening new sockets and establishing hands shaking with the server).
- andypants 14y ago
  christkv's comment is about creating a new connection per request. As I said, I agree, you should use a connection pool instead. When the web request comes in, you already have multiple database connections waiting for you. The connection overhead is not an issue then. And yes, that is exactly what I am thinking. More database connections = more concurrent database requests. I don't understand your reasoning about being in a single threaded environment. You made a statement but you haven't explained it. From what I can see, this problem is simply serial vs parallel. Is there something I'm missing? I don't see how the fact that node.js is single threaded changes the situation since the database requests aren't working within the node.js context. Edit: sorry, my previous reply may have been confusing. I was responding to your comment that having multiple connections is useless in a single-threaded environment. I was not suggesting that it may be helpful for the OP to make a new connection for each incoming http request.
  - joshguthrie 14y ago
    Why not try to benchmark the results and see how it really goes? :-) In both cases, the same process of reading/writing to one or more connection sockets happen, you send and receive the same content to MongoDb: you can not send twice as much queries, you're gonna send a query, THEN you're gonna send another, then you get an answer THEN you get another answer. Everything happens in the same sequence. Also take in account MongoDb's own concurrency properties.
    - andypants 14y ago
      Well then why would the nodejs mongodb driver support connection pooling?
      - christkv 14y ago
        for a very simple reason :) the current mongodb database has a thread pinned to each socket connection. This thread will serially execute the incoming wire protocol messages. So say you have 2 queries running at the same time. The first query takes 230 ms and the second one takes 100 ms. If you use only a single socket the first one will have to finish processing before the second one can be processed. This means that the total time of execution is 330 ms. If you fire each message on it's own socket they will be executed in parallel. The balance is between how big your pool is and how many messages are in flight. In the future this will change as MongoDB moves to some sort of eventloop or multiplexing of threads to sockets. But for the moment you have to strike a balance between operations in flight and how many sockets you open on the server. Another thing to keep in mind is that the more socket connections you open on the mongodb server the more memory needs to allocated on the heap and the more contexts switches between threads will happen.