7 ms·
...on a 32 physical cores machine.
by ogrisel 11y ago
...on a 32 physical cores machine.
- rodgerd 11y agoAnd? 32 cores are very affordable.
- taspeotis 11y agoThey're not always sitting idle waiting for one big OLAP query (well suited to parallel aggregation) to come in to exhibit a 25x speedup. Let's say you've got a web application with some cookie-cutter OLTP queries (which are well suited to serial plans ... no parallel aggregation benefit here). Your CPUs are running at 50% then some big OLAP query is going to come in and get maybe ... 13x faster. Which is a nice performance improvement, but not 25x. These numbers are a good validation of the work done in PostgreSQL, but you can't look at it and look at your X core machine, and say "my workload is going to be 25x faster!"
- ishi 11y agoThat's probably true for any benchmark - YMMV.
- pilif 11y ago> Let's say you've got a web application with some cookie-cutter OLTP queries (which are well suited to serial plans ... no parallel aggregation benefit here). in my case, the OLTP queries run against a dedicated slave that's mostly busy doing nothing, waiting for the next OLTP query to happen. So for me this will be very, very nice :-)
- taspeotis 11y agoSince I can't edit my post, the "25x speedup" I quote came from the original title of this post: "PostgreSQL Parallel Aggregate shows 25x speedup"
- mrits 11y agoIf your server is already running at 100% then there will be no speed increase!
- ogrisel 11y agoI was not trying to prove a point. It's just that giving a parallel speedup for a mostly CPU-bound task without mentioning the number of cores is not very interesting. 25x on 64 cores is not as interesting as 25x on 32 cores. If you know the number of cores you get that the efficiency is 80% (w.r.t. linear scaling) which is very good.
- eis 11y agoBut it is explained exactly and in detail how many CPUs they used. They even show a graph that shows performance for different number of CPUs.
- ogrisel 11y agothe blog post is very good, my comment was just about adding the missing info to the HN post title. Edit the title of the HN post has changed since I made this comment... probably by being merged with an earlier submission of the same URL. Now I understand why HN readers are misinterpreting my comment. I did not intend to be negative.
- eddd 11y agoI know ec2 people find it extraordinary having 32 cores, but on ovh you can get machine with 20c and 40 hyper threads and 256 GB of RAM for less than 350$ a month.
- dagw 11y agoI know ec2 people find it extraordinary having 32 cores How do you figure? As an 'EC2 person' I found that EC2 has made arbitrary specs utterly mundane and that is kind of the whole awesome point of EC2. Need 200GB of RAM for the afternoon to process some data? No problem, just press a button and a minute later you have 200GB of RAM for the afternoon to process some data. Then just turn it off when you are done and forget about it.
- eddd 11y agoI agree, but who spins up 32 cores for postgresql instance which is up 24/7?
- dagw 11y agoIf you need it, it's a bit over $1000/month. Which on the one hand is perhaps 2-3x more than renting a dedicated server. On the other hand it's only 2-3x times more than renting a dedicated server and if you have other stuff on AWS and make use of the rest of their features it might be worth while.
- felisml 11y agoYup. Small multiples of price inefficiency mostly don't bear thinking about until they group together in large batches. The time it takes your engineers to work out the difference is rarely worth it.
- merb 11y agoActually it's not really that expensive, when we buy dell machine's which costs like ~4000 Euro we get something like 16 core's. The most expensive thing in a server are prolly SAS disks.
- masklinn 11y agoYeah you can get 18 cores Xeon E7, plop that in a quad socket mobo and you've got 72 cores, 144 HT. They're expensive as hell but if that's what you need…
- lazyjones 11y ago... but it seems to scale linearly for 2-10 cores, isn't that good enough? Looks like a big win on very modest systems to me.