19 ms·
One of my favorites of these is only showing the speedup..."scales linearly", without comparing to a non-parallel baseline. First time I saw this was with Clik
by mpweiher 10y ago
One of my favorites of these is only showing the speedup..."scales linearly", without comparing to a non-parallel baseline. First time I saw this was with Clik, which reported linear scaling. I was impressed, until I found out that the parallel version was 6x slower on a single CPU than the sequential version.
Similar with a SPJ talk on parallel Haskell (can't find it right now). Big claims about this being the One True Way, and then required four cores to match C sequential case. Hmmm.
UPDATE: https://www.youtube.com/watch?v=NWSZ4c9yqW8 https://www.youtube.com/watch?v=NWSZ4c9yqW8 around the 1h 10min mark. Four cores to match sequential C (previously had written six, was wrong)
- adrianN 10y agoSee also "Scalability! But at What COST?" https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos...
- mpweiher 10y agoAhh, yes, the Configuration that Outperforms a Single Thread. I think this should be a standard + required measurement.
- ergl 10y agoI had a similar realization when I read on STM performance[1], where almost all benchmarks showed considerable slowdowns for the parallel+stm implementations when compared against sequential solutions. [1]: http://queue.acm.org/detail.cfm?id=1454466 http://queue.acm.org/detail.cfm?id=1454466
- inetknght 10y ago> This video is not available. > Sorry about that. Oh, Youtube...
- scott_s 10y agoAgreed. To my surprise, he kinda argues against this in one of his items (https://blogs.fau.de/hager/archives/6558 https://blogs.fau.de/hager/archives/6558): Always compare the fully parallel accelerator code with a sequential (single-core) CPU version. You can even argue in favor of this, saying that “serial CPU code constitutes a well-defined baseline.” Be sure to rush ahead to the next slide before anyone dares to ask why you don’t use a single GPGPU “core” to make the comparison even more “well-defined.” To be clear, this is about the performance of accelerated codes (such as GPU or FPGA). I had to read that multiple times to come up with the most charitable explanation. What he's really arguing against is framing: it's not valid to claim a 200x performance increase if your baseline is the sequential version on a single core. The graph he shows is actually about a 2.3x performance improvement (comparing GPU with ECC on against CPU with using 12 cores). But as mpweiher points out, I think you absolutely need to include the sequential version in your comparison set: the sequential performance is your "table stakes" baseline. If you can never beat it, then I don't care what your scalability on the device is! It should be there as a sanity check: yes, this approach is better than just using a single thread on the CPU. Where it gets interesting is that whether or not the sequential version wins is often dependent on problem size. In that case, the sequential version needs to be included because even if "large enough" problem sizes benefit from the accelerator, there may be real problem sizes that are not large enough. I think Hager is also coming from the world of HPC, where its okay to focus on particular kernels instead of end-to-end application performance. This is the case in the linked post; the performance graph is for a matrix multiply, not an application which uses matrix multiply for a larger result. And end-to-end application performance often requires factoring in some of the other bullet points (such as device communication) that focusing on kernels side-steps.
- smitherfield 10y agoAs a bit of a layman with this stuff, I'm wondering if "running the parallel version on a single CPU" means running a parallel algorithm (which is different from a sequential algorithm) on one CPU, or if the sequential and the single-threaded parallel algorithms are equivalent, and it's measuring the difference between e.g. 8 threads per 1 CPU vs. 1 thread per 8 CPUs (in which case, obviously, context switches would account for the performance difference).
- scott_s 10y agoWhen someone says "running the parallel version on a single CPU" they usually mean running the parallel version with 1 thread on 1 CPU. Comparing that to the sequential (non-parallel) version allows you to measure the baseline overhead cost of parallelization. There are also non-baseline overhead costs of parallelization, which usually come in the form of synchronization overhead between threads or processes.