5 ms·

Not everyone runs some PHP damaged setup where connection pooling has to be pushed out to a proxy. Some people use sane applications in the first place. There

Not everyone runs some PHP damaged setup where connection pooling has to be pushed out to a proxy. Some people use sane applications in the first place. There is no reason for those people to add an extra layer of latency and potential failure to achieve what they already have.

tbarbugli 13y ago
there are a number of applications that are sane and dont have a postgresql connection pooling built-in (eg. django introduced it on 1.6)
- nousernamesleft 13y ago
  Django requires multiple processes, which makes sharing things like connections quite difficult. That is not sane.
  - tbarbugli 13y ago
    whats your alternative to that?
    - nousernamesleft 13y ago
      A language without a global lock to make threads useless.
      - pekk 13y ago
        You have taken a FUD sound bite and repeated it without understanding. The language is not at issue here. CPython's GIL does not make threads useless at all, and there are already Python interpreters without a GIL anyway. An addiction to globally shared state across threads multiplies race conditions and scaling problems, even if the GIL magically goes away. If you are incapable of using even processes effectively then I really fear for you when you have to scale out horizontally. I'd say this more nicely if I could think of any way, but you don't know what you're talking about and you should read up before you begin talking about it again. It looks bad and you might mislead someone.
        nousernamesleft 13y ago
        No, you have taken a "stick my head in the sand and pretend the last 30 years of progress didn't happen" and repeating it without understanding. I do web development in haskell. I have thousands of concurrent connections to a single process, which is using 32 cores without problems. There has not been a single race condition, deadlock, mutex bottleneck, etc, etc. Just because you are happy with an absurdly primitive language, doesn't mean those of us in the 21st century are ignorant.
- marcosdumay 13y ago
  Django connections to the database are bounded by the number of threads and processess you set. It didn't have a pool, but it's a mistake to think it just oppened connections at will.
yummyfajitas 13y ago
True - it's only application insanity preventing a distributed collection of database clients from properly coordinating the proper allocation of postgres connections amongst themselves. Now true, in the event of a network partition, you lose the ability to open a database connection. But that's a small price to pay, right?
- teraflop 13y ago
  I guess this is sarcasm, but it's not clear what you're trying to actually say. How is a connection-pooling proxy supposed to make an application more resilient to network partitions? If anything, it's worse, since you have twice as many points of failure -- if either the application can't talk to the proxy, or the proxy can't talk to the database, you're hosed.
  - yummyfajitas 13y ago
    Practically speaking, you typically run pgbouncer on the same box as postgres. Even if you ran pgbouncer on a different box, the situation is still better. Suppose you have a probability P of having a partition between any two boxes. Then the probability of a partition taking down the system with pgbouncer is P. If you instead had N clients coordinating among themselves how many connections to use, your odds go way up. The probability of a partition existing between two clients is 1-(1-P)^(N(N-1)/2) ~= N(N-1)P/2 (for very small P). Once two clients can't communicate, neither one can connect to the DB (because they don't know if the other two have pushed the db over the limit). All clients will need to stop connecting to the DB if any client is ever disconnected from the other N-1 of them. In a distributed system, ensuring that SUM(x[i]) remains bounded for all time is a tricky problem to solve.
    - wmf 13y ago
      Or you could give each client C/N connections.
      - yummyfajitas 13y ago

tbarbugli 13y ago

there are a number of applications that are sane and dont have a postgresql connection pooling built-in (eg. django introduced it on 1.6)

nousernamesleft 13y ago
Django requires multiple processes, which makes sharing things like connections quite difficult. That is not sane.
- tbarbugli 13y ago
  whats your alternative to that?
  - nousernamesleft 13y ago
    A language without a global lock to make threads useless.
    - pekk 13y ago
      You have taken a FUD sound bite and repeated it without understanding. The language is not at issue here. CPython's GIL does not make threads useless at all, and there are already Python interpreters without a GIL anyway. An addiction to globally shared state across threads multiplies race conditions and scaling problems, even if the GIL magically goes away. If you are incapable of using even processes effectively then I really fear for you when you have to scale out horizontally. I'd say this more nicely if I could think of any way, but you don't know what you're talking about and you should read up before you begin talking about it again. It looks bad and you might mislead someone.
      - nousernamesleft 13y ago
        No, you have taken a "stick my head in the sand and pretend the last 30 years of progress didn't happen" and repeating it without understanding. I do web development in haskell. I have thousands of concurrent connections to a single process, which is using 32 cores without problems. There has not been a single race condition, deadlock, mutex bottleneck, etc, etc. Just because you are happy with an absurdly primitive language, doesn't mean those of us in the 21st century are ignorant.
marcosdumay 13y ago
Django connections to the database are bounded by the number of threads and processess you set. It didn't have a pool, but it's a mistake to think it just oppened connections at will.

yummyfajitas 13y ago

True - it's only application insanity preventing a distributed collection of database clients from properly coordinating the proper allocation of postgres connections amongst themselves. Now true, in the event of a network partition, you lose the ability to open a database connection. But that's a small price to pay, right?

teraflop 13y ago
I guess this is sarcasm, but it's not clear what you're trying to actually say. How is a connection-pooling proxy supposed to make an application more resilient to network partitions? If anything, it's worse, since you have twice as many points of failure -- if either the application can't talk to the proxy, or the proxy can't talk to the database, you're hosed.
- yummyfajitas 13y ago
  Practically speaking, you typically run pgbouncer on the same box as postgres. Even if you ran pgbouncer on a different box, the situation is still better. Suppose you have a probability P of having a partition between any two boxes. Then the probability of a partition taking down the system with pgbouncer is P. If you instead had N clients coordinating among themselves how many connections to use, your odds go way up. The probability of a partition existing between two clients is 1-(1-P)^(N(N-1)/2) ~= N(N-1)P/2 (for very small P). Once two clients can't communicate, neither one can connect to the DB (because they don't know if the other two have pushed the db over the limit). All clients will need to stop connecting to the DB if any client is ever disconnected from the other N-1 of them. In a distributed system, ensuring that SUM(x[i]) remains bounded for all time is a tricky problem to solve.
  - wmf 13y ago
    Or you could give each client C/N connections.
    - yummyfajitas 13y ago