5 ms·

Does anyone know if PostgreSQL has this similar multicore bottleneck?

mdasen 18y ago
PostgreSQL is much better at using multiple cores, but it doesn't matter (at least not for web applications). Web applications don't have a problem handling one person. They have a problem handling 1,000 people who all want to load a page at the same time. That means that you're not worried about using all your server resources for one query. You're going to have 1,000 queries and so 250 go to each core in a quad-core box and you don't care that MySQL (InnoDB really) doesn't do the multi-core thing because it's not like PostgreSQL would benefit from it since it's just easier to do different queries using a different core. Multi-core capabilities could help when you are running fewer concurrent queries than you have processor cores. However, once you get to 4-8 concurrent users, you don't need to worry about it so much because you'll saturate all the cores through the different queries and when you're below 4-8 concurrent users you don't need to worry about it because it's easy to serve few users. Now, for data warehousing and other applications, multi-core capabilities can be the difference between a query taking 1 hour and a query taking 4 hours. But that's not a web application. In fact, using MySQL for data warehousing is just a bad idea. The author is totally right that you shouldn't just expect technology to save you (and I personally think DHH is taking the "wait until it is a problem. if becomes a problem and new hardware can solve it, do that. if not, then deal with it" approach rather than simply expecting a solution). However, web applications are about running many things at once, not running one big thing. P.S. Databases usually aren't CPU constrained. It's almost always memory or disk that slows you down. Spend the money on more RAM or better disks and don't worry about the CPU so much. P.P.S. Both MySQL and PostgreSQL are wonderful databases for web applications so let's not turn this into some silly flame war, please?
- Retric 18y ago
  Your ignoring the issue of locking, which can effectively transform your multi core CPU in a single core. Secondly databases can be limited depending on what your asking them to do not just what type of work your doing. IMO the largest advantage to multi core CPU's is handing a longer list of requests the the disk controller so it can better optimize the the head's path.
- sounddust 18y ago
  PostgreSQL is much better at using multiple cores, and it absolutely does matter for web applications. When you have 1,000 people who all want to load a page at the same time, you're only going to hit the database once for that. What you will have instead is 5,000 people who want to go to 1,000 different pages, some of them updating data and some of them retrieving it, and you will have 1,000 different queries that are either selects or updates. Now, PostgreSQL was designed from the start to be robust enough to handle this type of concurrency - to allow the minimum amount of data to be locked while updating, to free the selects from being blocked by those writes. As a result of being built on such a solid foundation, it has been easy to optimize over the years, and has supported the emergence of commmon multi-core computing quite well. MySQL, on the other hand, was designed to return queries extremely quickly, with as little getting in the way of doing so as possible. Unfortunately, their model did not scale as well. When you have 500 reads and 1 write, and you lock the entire table for that write, it's not a big deal because the write takes no time and you're unblocked again. But when you have an array of hundreds of simultaneous reads and writes, it becomes a mess if you don't have fine-grained locking capabilities. That having been said, MySQL is progressing as well. But I think (not trying to start a flame war) that PostgreSQL did things slow and right in the beginning and it's paying off pretty well. Finally, I disagree that databases are not CPU constrained. Any website with decent traffic will have the database entirely cached in RAM at all times, with the only disk activity being the journaling.
  - mdasen 18y ago
    PostgreSQL was designed "correct" from the start. However, MySQL does many of the same things today. You start by saying you'll have 5,000 people who want 1,000 different pages with some updates, some selects, etc. Well, once you have more queries than cores, the benefit of splitting work over multiple cores is gone since none of the queries can logically use even a whole single core given the level of concurrency. You go on to say that PostgreSQL was designed to eliminate unnecessary locking so that selects wouldn't be blocked. MySQL's MyISAM database doesn't support this, that is correct. However, InnoDB does support the same MVCC model that PostgreSQL uses and likewise eliminates the locking issue. Your argument rests on MySQL locking an entire table to do a write: "When you have 500 reads and 1 write, and you lock the entire table for that write, it's not a big deal. . . But when you have an array of hundreds of simultaneous reads and writes, it becomes a mess if you don't have fine-grained locking capabilities." A great article on this problem is here: http://www.onlamp.com/pub/a/onlamp/2001/05/25/postgresql_mvcc.html http://www.onlamp.com/pub/a/onlamp/2001/05/25/postgresql_mvc.... It's great that PostgreSQL supports that, but MySQL supports it too today (http://wiki.oracle.com/page/InnoDB?t=anon http://wiki.oracle.com/page/InnoDB?t=anon). There are plenty of things that one can say are issues between the two. MySQL's inferior query planner. The fact that PostgreSQL can suggest indexes based on query history. PostgreSQL's weak replication (Slony-I's communication costs grow quadratically, yuck!). MySQL's acceptance of February 31st as a real date. MySQL's poor subquery optimization. PostgreSQL's more limited data partitioning. They aren't equal in all ways and PostgreSQL is a wonderful database, but saying that MySQL needs to lock a table to do a write is just wrong in a very partisan manner. I've never really understood such partisanship. Knowing the strengths and weaknesses of multiple products makes you aware of what is good for a project and what isn't. Even better, once you're using one of them, you know what to do and what to avoid with it. Not confronting the reality of how alternative systems work just means that the chance of picking the best system is more luck than information. For what it's worth, I use PostgreSQL in my personal projects. It's great. However, it's also important to understand that MySQL of 2008 is not MySQL of 2001. It's come a long way in the "correctness" camp and the old arguments about Multi-Version Concurrency Control don't apply anymore. Oh, and from Power PostgreSQL, Disks > RAM > CPU (http://www.powerpostgresql.com/PerfList http://www.powerpostgresql.com/PerfList).
andrewf 18y ago
Sun has previously demonstrated Postgres scaling well on their 32-way (8 cores x 4 threads/core) hardware. http://it.toolbox.com/blogs/database-soup/postgresql-publishes-first-real-benchmark-17470 http://it.toolbox.com/blogs/database-soup/postgresql-publish...

PostgreSQL is much better at using multiple cores, but it doesn't matter (at least not for web applications). Web applications don't have a problem handling one person. They have a problem handling 1,000 people who all want to load a page at the same time. That means that you're not worried about using all your server resources for one query. You're going to have 1,000 queries and so 250 go to each core in a quad-core box and you don't care that MySQL (InnoDB really) doesn't do the multi-core thing because it's not like PostgreSQL would benefit from it since it's just easier to do different queries using a different core. Multi-core capabilities could help when you are running fewer concurrent queries than you have processor cores. However, once you get to 4-8 concurrent users, you don't need to worry about it so much because you'll saturate all the cores through the different queries and when you're below 4-8 concurrent users you don't need to worry about it because it's easy to serve few users. Now, for data warehousing and other applications, multi-core capabilities can be the difference between a query taking 1 hour and a query taking 4 hours. But that's not a web application. In fact, using MySQL for data warehousing is just a bad idea. The author is totally right that you shouldn't just expect technology to save you (and I personally think DHH is taking the "wait until it is a problem. if becomes a problem and new hardware can solve it, do that. if not, then deal with it" approach rather than simply expecting a solution). However, web applications are about running many things at once, not running one big thing. P.S. Databases usually aren't CPU constrained. It's almost always memory or disk that slows you down. Spend the money on more RAM or better disks and don't worry about the CPU so much. P.P.S. Both MySQL and PostgreSQL are wonderful databases for web applications so let's not turn this into some silly flame war, please?

Retric 18y ago
Your ignoring the issue of locking, which can effectively transform your multi core CPU in a single core. Secondly databases can be limited depending on what your asking them to do not just what type of work your doing. IMO the largest advantage to multi core CPU's is handing a longer list of requests the the disk controller so it can better optimize the the head's path.
sounddust 18y ago
PostgreSQL is much better at using multiple cores, and it absolutely does matter for web applications. When you have 1,000 people who all want to load a page at the same time, you're only going to hit the database once for that. What you will have instead is 5,000 people who want to go to 1,000 different pages, some of them updating data and some of them retrieving it, and you will have 1,000 different queries that are either selects or updates. Now, PostgreSQL was designed from the start to be robust enough to handle this type of concurrency - to allow the minimum amount of data to be locked while updating, to free the selects from being blocked by those writes. As a result of being built on such a solid foundation, it has been easy to optimize over the years, and has supported the emergence of commmon multi-core computing quite well. MySQL, on the other hand, was designed to return queries extremely quickly, with as little getting in the way of doing so as possible. Unfortunately, their model did not scale as well. When you have 500 reads and 1 write, and you lock the entire table for that write, it's not a big deal because the write takes no time and you're unblocked again. But when you have an array of hundreds of simultaneous reads and writes, it becomes a mess if you don't have fine-grained locking capabilities. That having been said, MySQL is progressing as well. But I think (not trying to start a flame war) that PostgreSQL did things slow and right in the beginning and it's paying off pretty well. Finally, I disagree that databases are not CPU constrained. Any website with decent traffic will have the database entirely cached in RAM at all times, with the only disk activity being the journaling.
- mdasen 18y ago
  PostgreSQL was designed "correct" from the start. However, MySQL does many of the same things today. You start by saying you'll have 5,000 people who want 1,000 different pages with some updates, some selects, etc. Well, once you have more queries than cores, the benefit of splitting work over multiple cores is gone since none of the queries can logically use even a whole single core given the level of concurrency. You go on to say that PostgreSQL was designed to eliminate unnecessary locking so that selects wouldn't be blocked. MySQL's MyISAM database doesn't support this, that is correct. However, InnoDB does support the same MVCC model that PostgreSQL uses and likewise eliminates the locking issue. Your argument rests on MySQL locking an entire table to do a write: "When you have 500 reads and 1 write, and you lock the entire table for that write, it's not a big deal. . . But when you have an array of hundreds of simultaneous reads and writes, it becomes a mess if you don't have fine-grained locking capabilities." A great article on this problem is here: http://www.onlamp.com/pub/a/onlamp/2001/05/25/postgresql_mvcc.html http://www.onlamp.com/pub/a/onlamp/2001/05/25/postgresql_mvc.... It's great that PostgreSQL supports that, but MySQL supports it too today (http://wiki.oracle.com/page/InnoDB?t=anon http://wiki.oracle.com/page/InnoDB?t=anon). There are plenty of things that one can say are issues between the two. MySQL's inferior query planner. The fact that PostgreSQL can suggest indexes based on query history. PostgreSQL's weak replication (Slony-I's communication costs grow quadratically, yuck!). MySQL's acceptance of February 31st as a real date. MySQL's poor subquery optimization. PostgreSQL's more limited data partitioning. They aren't equal in all ways and PostgreSQL is a wonderful database, but saying that MySQL needs to lock a table to do a write is just wrong in a very partisan manner. I've never really understood such partisanship. Knowing the strengths and weaknesses of multiple products makes you aware of what is good for a project and what isn't. Even better, once you're using one of them, you know what to do and what to avoid with it. Not confronting the reality of how alternative systems work just means that the chance of picking the best system is more luck than information. For what it's worth, I use PostgreSQL in my personal projects. It's great. However, it's also important to understand that MySQL of 2008 is not MySQL of 2001. It's come a long way in the "correctness" camp and the old arguments about Multi-Version Concurrency Control don't apply anymore. Oh, and from Power PostgreSQL, Disks > RAM > CPU (http://www.powerpostgresql.com/PerfList http://www.powerpostgresql.com/PerfList).

andrewf 18y ago

Sun has previously demonstrated Postgres scaling well on their 32-way (8 cores x 4 threads/core) hardware. http://it.toolbox.com/blogs/database-soup/postgresql-publishes-first-real-benchmark-17470 http://it.toolbox.com/blogs/database-soup/postgresql-publish...