7 ms·

Multi-core and multi-threading performance

67 points by pmarin 14y ago

stephengillie 14y ago
The Megahertz-myth was a result of people being conditioned to see the clockspeed as an absolute measure of performance. It's more a symptom of people looking for one absolute measure of performance, because it's simply easier (less computationally expensive for the human brain) than dealing with a multi-variable valuation problem. IQ is another example, it's easier to deal with just that number instead of considering the whole person. Engine displacement (ie. 2.2L engine) is another - there's a range of variables such as max torque and max horsepower, and which engine speed maximizes those, etc.
marshray 14y ago
Amdhal "The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program" was talking about single programs which at that time were generally batch processing applications. Just like today, developers tended to start with single-threaded code and then looked for ways to parallelize it. When he says "speedup...is limited", he means that in the mathematical sense. I.e., if you had infinite processors the program would not take zero time to complete. Adding cores is still great when you have many threads and processes competing for CPU time. If your workload is blocking on network or disk, it means you have enough cores. Anyone who would expect additional cores to significantly speed up a single, single-threaded, application is simply in error. If this is a myth, I don't know anyone who believes it.
- stephengillie 14y ago
  Anyone who would expect additional cores to significantly speed up a single, single-threaded, application is simply in error. </ snip> If this is a myth, I don't know anyone who believes it. 9 women can't create an infant in a month?
  - marshray 14y ago
    Ah yes, The Mythical Woman-Month. I stand corrected. :-)
  - adrianN 14y ago
    Once the pipeline is warmed up...
- jbooth 14y ago
  The point is that even the most parallel programs tend to have bottlenecks where only one thread can proceed. As you add more cores, these bottlenecks increase in relative importance to the parallel parts, which are easily dealt with by your additional parallel capacity.
  - marshray 14y ago
    Did anyone (who writes parallel code or purchases CPUs professionally) really not know this? I'm not trying to pick on the article, I like it overall. I took an interest in the Anand article when I learned you could build a 64-core box for about $4000. This is the article it's in response to http://www.anandtech.com/show/5057/the-bulldozer-aftermath-delving-even-deeper http://www.anandtech.com/show/5057/the-bulldozer-aftermath-d... the crux of which is: Some, including sources inside AMD, have blamed Global Foundries for not delivering higher clocked SKUs. Sure, the clock speed targets for Interlagos were probably closer to 3GHz instead of 2.3GHz. But that does not explain why the extra integer cores do not deliver. We were promised up to 50% higher performance thanks to the 33% extra cores, but we got 20% at the most. So the real questions seem to be: Is Interlagos' perfomance disappointing on pure-parallel benchmarks as well, or just those with a noticeable serial bottleneck? If there is poor scalability, what is the cause? and perhaps most importantly: Will my kind of workload actually benefit from Interlagos' more cores? I think that probably what's happening is that even the pure parallel benchmarks are not scaling linearly relative to the previous generation of chips, the reason is a combination of architectural changes in the core itself and memory bus contention, and the target market for these chips is VPS hosts running hundreds of VM images.
    - jbooth 14y ago
      I agree with everything you're saying, I just think the 'obvious point' is one that's papered over in the public discourse a lot because it's not exciting. It is interesting why they don't scale linearly and I agree that it's probably a combination of memory bus contention and/or the chip itself being a little slower. The whole bulldozer thing seemed like a marketing stunt rather than thoughtful pursuit of higher performance to me. Given the frequency and cost of cache miss rates, Intel HT seems to get the same work done on fewer watts.
deleted 14y ago
[deleted]
- marshray 14y ago
  It probably has to do with the desire to keep gameplay very consistent across all users' hardware. They don't want to give a huge advantage to the enthusiast with a dozen cores over the person playing on their laptop. But making the graphics better is something that doesn't seem to help your score much. More realistic fog may even make the game a little harder. So it's probably no coincidence that GPUs are leading the way in parallelism.
  - stephengillie 14y ago
    Sorry, if I realized you were replying, I wouldn't have deleted :p While videogames have been one of the main economic forces pushing processor technology, I've heard that few games can actually use more than 1 CPU core. Is this because of a difference in the way games are programmed? Is it because single-core programming is (I've heard) so much easier, and games are more often written by aspiring novices than professional drones?
    - maximilianburke 14y ago
      That's not true, at least for AAA games. Most modern AAA games make good use of all available processors. This is especially important for games that ship on the Xbox 360 and PS3 because these devices only start delivering impressive results when you go parallel. It's difficult to come up with a good multicore game engine design and even more difficult to do it on the first try. What I've observed over the years is that these AAA games are steadily becoming more and more parallelizable as they are iterated on. An architectural pattern that seems to be working well these days is a job/task-based one where the program is expressed as a graph with the tasks (well defined units of work) are connected via a network of dependencies. This model works well because it doesn't force you into requiring certain hardware configurations, it scales well from single core phones up to 32+ core workstations. Now, whether or not your content can scale as well is another question :)
  - 14y ago
rheide 14y ago
Very good points. I remember playing Need for Speed 2 SE on a Pentium II 200MHz with an mp3 playing in the background with no problems at all. These days we can emulate several of those machines inside a single OS, while browsing the web and watching a movie at the same time.
CookWithMe 14y ago
tl;dr Use the right tool for the job. You need to know your tool (CPU) and your job (application(s)). I.e. an app which is heavily affected by Amdahls law doesn't really profit from more cores, where as highly parallel apps do profit a lot (see reference to Niagara architecture at the end). What I find strange is that the article first introduces the difference between multitasking and multithreading in a rather lengthy manner, and then uses Amdahl's law, which only applies to multithreading. Especially, there is a gap on what is stated first: > "So multitasking means you are using multiple applications at the same time. Which you always do, these days." With the conclusion: > "With multi-core, your mileage may vary, depending on the algorithms used by the application (to what extent are they parallelized, and to what extent do they have to remain sequential?), and on the performance of the individual cores." His discussion is all well when you have one app that is really needing the power and the other multitasking apps don't need much CPU (e.g. playing mp3 and such). Which is the case for most desktop systems, I'd say. For servers, it can be really different, and you could either have many processes running or one process is heavily optimized for multithreading and avoids Amdahl's law nicely. Then the discussion is kinda void.
- jacquesm 14y ago
  You don't avoid Amdahls law nicely or otherwise for those problems to which it applies. It's a mathematical thing, not a technical thing. Problems that are split up over many cores, processes or even clusters of machines always have some serial component, and that serial component determines the minimum run-time for the whole computation. So even if all the rest of the work would be reduced to '0' then there would still be that serial component left, whatever it was. This is not at all about playing mp3s or server loads such as web serving that are embarrassingly parallel and that can be run in a shared nothing environment. It is all about computing solutions to problems where there is some shared state between the various parts of the computation. In computations like that the serial bits are visible as bottle-necks in the data flow, either because there is a synchronization point and some kind of reduction of the data that can only be done in a serial fashion, an ordering or some other operation that can only be done by a limited number of the total available cores. Those sections, and only those sections will put a lower limit on how fast you can solve that particular problem for a given data set.
CookWithMe 14y ago
The biggest myth is probably that the CPU is the most important part for performance. There are so many other bottlenecks that you can easily run into (drive, memory, memory bus, ...) that blindly updating the CPU (more MHz / more cores) is simply wrong.
- majorlazer 14y ago
  I think it's pretty incredible to see our processors finally catch up and overtake our computing requirements. If you have a first-gen i5, or even a Core 2 Duo, your CPU is plenty fast to complete 99% of computer tasks out there without feeling any sluggishness. 10 years ago, when people would come to me for advice on purchasing a laptop/desktop, I would tell them to buy the best processor they can afford and then upgrade the rest of the components while they use it. Nowadays, most decent laptops ship with an i5 at minimum, some people ask me if they should upgrade to the i7. I always tell them to save their money and spend it on a faster SSD or more ram (if the ram is <4gb).
- ww520 14y ago
  The biggest improvement on current laptop for me is SSD. Man that's biggest jump in performance (perceptive) in a long while.
frozenport 14y ago
There is an untold story involving software licenses. Many software are licensed to run on certain PE (cpu core) counts. A 1024 PE license costs you 64 times as much as single host license. Moreover the software costs may easily exceed the core costs. In these cases the bottleneck is software fees and the easiest solution is to use higher quality cores.
lutze 14y ago
Nice little shoutout to the Amiga there, and why it was such a groundbreaking machine! Nothing else a home user could buy could do pre-emptive multitasking in 1985... Or even 1995 AFAIK.
- jacquesm 14y ago
  There was the 'unicorn'. http://en.wikipedia.org/wiki/Torch_Computers http://en.wikipedia.org/wiki/Torch_Computers based on a 68010.
  - lutze 14y ago
    Well, Unix would do it. Honestly never heard of these though, thanks.
systematical 14y ago
Very interesting read. So if clock speed (when analyzing different architectures) doesn't matter, what should I be looking at?
scarmig 14y ago
I've got a question that I'd love for someone to provide feedback on. Is it the case that a multicore processor must have identical cores? Let's say there are three tasks, A, B, and C. C depends on A and B, but A is 4x as costly as B. Adding more cores doesn't get around that inherently sequential part, so because of overhead and cost considerations it isn't worth it to parallelize. This applies to most computational tasks: as you add more cores, the effect of the sequential part dominates more and more and you get decreasing benefits. But what if you have one beefy processor to deal with the heavily sequential parts, and a multitude of less impressive ones to deal with the easily parallelizable parts? Doesn't that give you the best of both worlds? Is it technically infeasible for some reason?
- shin_lao 14y ago
  What do you mean by a "less beefy CPU"? Once you've been able to design the new CPU and test it, it doesn't make sense to have an older/slower design. You could reduce the cache, but it would probably more expensive to build. If one of your core is idle, the CPU will load it, if there is work to be done.
  - bornhuetter 14y ago
    I think what scarmig is getting at is - wouldn't it be better to have 1 2ghz core and 3 1ghz cores, rather than 4 1.5 ghz cores. I don't know the answer myself, and would be interested if someone could clarify why Intel don't make quad cores with one core being faster than the others. My best guess is that it's hard to keep things in sync if you have different cores on different frequencies or multipliers.
- paol 14y ago
  Yes, that is feasible and there are real systems that take that approach (millions of them in fact - look up the architecture of the Sony Playstation 3 console if you're curious). One problem that surfaces is that such asymmetric systems are harder to write software for, which can make them a bit unpopular.
- CookWithMe 14y ago
humdumb 14y ago
I liked this blog entry a lot. A wee bit of multi-core/threads anti-hype is healthy. The comment about people wanting a single absolute measure of something is also spot on. The biggest bottlenecks to me were (and still are) memory and the network. Mult-cores are nice, but given the choice I'd still trade additional CPU's for more memory or a faster network.

67 points by pmarin 14y ago

The Megahertz-myth was a result of people being conditioned to see the clockspeed as an absolute measure of performance. It's more a symptom of people looking for one absolute measure of performance, because it's simply easier (less computationally expensive for the human brain) than dealing with a multi-variable valuation problem. IQ is another example, it's easier to deal with just that number instead of considering the whole person. Engine displacement (ie. 2.2L engine) is another - there's a range of variables such as max torque and max horsepower, and which engine speed maximizes those, etc.

marshray 14y ago

Amdhal "The speedup of a program using multiple processors in parallel computing is limited by the time needed for the sequential fraction of the program" was talking about single programs which at that time were generally batch processing applications. Just like today, developers tended to start with single-threaded code and then looked for ways to parallelize it. When he says "speedup...is limited", he means that in the mathematical sense. I.e., if you had infinite processors the program would not take zero time to complete. Adding cores is still great when you have many threads and processes competing for CPU time. If your workload is blocking on network or disk, it means you have enough cores. Anyone who would expect additional cores to significantly speed up a single, single-threaded, application is simply in error. If this is a myth, I don't know anyone who believes it.

stephengillie 14y ago
Anyone who would expect additional cores to significantly speed up a single, single-threaded, application is simply in error. </ snip> If this is a myth, I don't know anyone who believes it. 9 women can't create an infant in a month?
- marshray 14y ago
  Ah yes, The Mythical Woman-Month. I stand corrected. :-)
- adrianN 14y ago
  Once the pipeline is warmed up...
jbooth 14y ago
The point is that even the most parallel programs tend to have bottlenecks where only one thread can proceed. As you add more cores, these bottlenecks increase in relative importance to the parallel parts, which are easily dealt with by your additional parallel capacity.
- marshray 14y ago
  Did anyone (who writes parallel code or purchases CPUs professionally) really not know this? I'm not trying to pick on the article, I like it overall. I took an interest in the Anand article when I learned you could build a 64-core box for about $4000. This is the article it's in response to http://www.anandtech.com/show/5057/the-bulldozer-aftermath-delving-even-deeper http://www.anandtech.com/show/5057/the-bulldozer-aftermath-d... the crux of which is: Some, including sources inside AMD, have blamed Global Foundries for not delivering higher clocked SKUs. Sure, the clock speed targets for Interlagos were probably closer to 3GHz instead of 2.3GHz. But that does not explain why the extra integer cores do not deliver. We were promised up to 50% higher performance thanks to the 33% extra cores, but we got 20% at the most. So the real questions seem to be: Is Interlagos' perfomance disappointing on pure-parallel benchmarks as well, or just those with a noticeable serial bottleneck? If there is poor scalability, what is the cause? and perhaps most importantly: Will my kind of workload actually benefit from Interlagos' more cores? I think that probably what's happening is that even the pure parallel benchmarks are not scaling linearly relative to the previous generation of chips, the reason is a combination of architectural changes in the core itself and memory bus contention, and the target market for these chips is VPS hosts running hundreds of VM images.
  - jbooth 14y ago
    I agree with everything you're saying, I just think the 'obvious point' is one that's papered over in the public discourse a lot because it's not exciting. It is interesting why they don't scale linearly and I agree that it's probably a combination of memory bus contention and/or the chip itself being a little slower. The whole bulldozer thing seemed like a marketing stunt rather than thoughtful pursuit of higher performance to me. Given the frequency and cost of cache miss rates, Intel HT seems to get the same work done on fewer watts.

deleted 14y ago

[deleted]

marshray 14y ago
It probably has to do with the desire to keep gameplay very consistent across all users' hardware. They don't want to give a huge advantage to the enthusiast with a dozen cores over the person playing on their laptop. But making the graphics better is something that doesn't seem to help your score much. More realistic fog may even make the game a little harder. So it's probably no coincidence that GPUs are leading the way in parallelism.
- stephengillie 14y ago
  Sorry, if I realized you were replying, I wouldn't have deleted :p While videogames have been one of the main economic forces pushing processor technology, I've heard that few games can actually use more than 1 CPU core. Is this because of a difference in the way games are programmed? Is it because single-core programming is (I've heard) so much easier, and games are more often written by aspiring novices than professional drones?
  - maximilianburke 14y ago
    That's not true, at least for AAA games. Most modern AAA games make good use of all available processors. This is especially important for games that ship on the Xbox 360 and PS3 because these devices only start delivering impressive results when you go parallel. It's difficult to come up with a good multicore game engine design and even more difficult to do it on the first try. What I've observed over the years is that these AAA games are steadily becoming more and more parallelizable as they are iterated on. An architectural pattern that seems to be working well these days is a job/task-based one where the program is expressed as a graph with the tasks (well defined units of work) are connected via a network of dependencies. This model works well because it doesn't force you into requiring certain hardware configurations, it scales well from single core phones up to 32+ core workstations. Now, whether or not your content can scale as well is another question :)
- 14y ago

rheide 14y ago

Very good points. I remember playing Need for Speed 2 SE on a Pentium II 200MHz with an mp3 playing in the background with no problems at all. These days we can emulate several of those machines inside a single OS, while browsing the web and watching a movie at the same time.

CookWithMe 14y ago

tl;dr Use the right tool for the job. You need to know your tool (CPU) and your job (application(s)). I.e. an app which is heavily affected by Amdahls law doesn't really profit from more cores, where as highly parallel apps do profit a lot (see reference to Niagara architecture at the end). What I find strange is that the article first introduces the difference between multitasking and multithreading in a rather lengthy manner, and then uses Amdahl's law, which only applies to multithreading. Especially, there is a gap on what is stated first: > "So multitasking means you are using multiple applications at the same time. Which you always do, these days." With the conclusion: > "With multi-core, your mileage may vary, depending on the algorithms used by the application (to what extent are they parallelized, and to what extent do they have to remain sequential?), and on the performance of the individual cores." His discussion is all well when you have one app that is really needing the power and the other multitasking apps don't need much CPU (e.g. playing mp3 and such). Which is the case for most desktop systems, I'd say. For servers, it can be really different, and you could either have many processes running or one process is heavily optimized for multithreading and avoids Amdahl's law nicely. Then the discussion is kinda void.

jacquesm 14y ago
You don't avoid Amdahls law nicely or otherwise for those problems to which it applies. It's a mathematical thing, not a technical thing. Problems that are split up over many cores, processes or even clusters of machines always have some serial component, and that serial component determines the minimum run-time for the whole computation. So even if all the rest of the work would be reduced to '0' then there would still be that serial component left, whatever it was. This is not at all about playing mp3s or server loads such as web serving that are embarrassingly parallel and that can be run in a shared nothing environment. It is all about computing solutions to problems where there is some shared state between the various parts of the computation. In computations like that the serial bits are visible as bottle-necks in the data flow, either because there is a synchronization point and some kind of reduction of the data that can only be done in a serial fashion, an ordering or some other operation that can only be done by a limited number of the total available cores. Those sections, and only those sections will put a lower limit on how fast you can solve that particular problem for a given data set.

CookWithMe 14y ago

The biggest myth is probably that the CPU is the most important part for performance. There are so many other bottlenecks that you can easily run into (drive, memory, memory bus, ...) that blindly updating the CPU (more MHz / more cores) is simply wrong.

majorlazer 14y ago
I think it's pretty incredible to see our processors finally catch up and overtake our computing requirements. If you have a first-gen i5, or even a Core 2 Duo, your CPU is plenty fast to complete 99% of computer tasks out there without feeling any sluggishness. 10 years ago, when people would come to me for advice on purchasing a laptop/desktop, I would tell them to buy the best processor they can afford and then upgrade the rest of the components while they use it. Nowadays, most decent laptops ship with an i5 at minimum, some people ask me if they should upgrade to the i7. I always tell them to save their money and spend it on a faster SSD or more ram (if the ram is <4gb).
ww520 14y ago
The biggest improvement on current laptop for me is SSD. Man that's biggest jump in performance (perceptive) in a long while.

frozenport 14y ago

There is an untold story involving software licenses. Many software are licensed to run on certain PE (cpu core) counts. A 1024 PE license costs you 64 times as much as single host license. Moreover the software costs may easily exceed the core costs. In these cases the bottleneck is software fees and the easiest solution is to use higher quality cores.

lutze 14y ago

Nice little shoutout to the Amiga there, and why it was such a groundbreaking machine! Nothing else a home user could buy could do pre-emptive multitasking in 1985... Or even 1995 AFAIK.

jacquesm 14y ago
There was the 'unicorn'. http://en.wikipedia.org/wiki/Torch_Computers http://en.wikipedia.org/wiki/Torch_Computers based on a 68010.
- lutze 14y ago
  Well, Unix would do it. Honestly never heard of these though, thanks.

systematical 14y ago

Very interesting read. So if clock speed (when analyzing different architectures) doesn't matter, what should I be looking at?

scarmig 14y ago

I've got a question that I'd love for someone to provide feedback on. Is it the case that a multicore processor must have identical cores? Let's say there are three tasks, A, B, and C. C depends on A and B, but A is 4x as costly as B. Adding more cores doesn't get around that inherently sequential part, so because of overhead and cost considerations it isn't worth it to parallelize. This applies to most computational tasks: as you add more cores, the effect of the sequential part dominates more and more and you get decreasing benefits. But what if you have one beefy processor to deal with the heavily sequential parts, and a multitude of less impressive ones to deal with the easily parallelizable parts? Doesn't that give you the best of both worlds? Is it technically infeasible for some reason?

shin_lao 14y ago
What do you mean by a "less beefy CPU"? Once you've been able to design the new CPU and test it, it doesn't make sense to have an older/slower design. You could reduce the cache, but it would probably more expensive to build. If one of your core is idle, the CPU will load it, if there is work to be done.
- bornhuetter 14y ago
  I think what scarmig is getting at is - wouldn't it be better to have 1 2ghz core and 3 1ghz cores, rather than 4 1.5 ghz cores. I don't know the answer myself, and would be interested if someone could clarify why Intel don't make quad cores with one core being faster than the others. My best guess is that it's hard to keep things in sync if you have different cores on different frequencies or multipliers.
paol 14y ago
Yes, that is feasible and there are real systems that take that approach (millions of them in fact - look up the architecture of the Sony Playstation 3 console if you're curious). One problem that surfaces is that such asymmetric systems are harder to write software for, which can make them a bit unpopular.
CookWithMe 14y ago

humdumb 14y ago

I liked this blog entry a lot. A wee bit of multi-core/threads anti-hype is healthy. The comment about people wanting a single absolute measure of something is also spot on. The biggest bottlenecks to me were (and still are) memory and the network. Mult-cores are nice, but given the choice I'd still trade additional CPU's for more memory or a faster network.