4 ms·

When you have tens of CPU cores, you really do start to run up against Amdahl's Law and soon don't get the speedup that you might expect. e.g. let's say 90% of

When you have tens of CPU cores, you really do start to run up against Amdahl's Law and soon don't get the speedup that you might expect. e.g. let's say 90% of your task can be perfectly parallelised. The remaining 10% of the time will not be improved. So you won't ever get more than a 10x speedup no matter how many tens or hundreds of extra CPU cores you have...

xienze 12y ago
Well, you're thinking in terms of parallelising a single task. For a server, you're talking about a large number of concurrent tasks (users), so more cores are better in that case.
- joosters 12y ago
  That's basically a truism. If you have parallelisable tasks, then they will parallelise. No surprise there. I was trying to make the point that as we step up the number of cores, the actual gain is less and less.
  - deleted 12y ago
    [deleted]
  - rbanffy 12y ago
    Again, as it was pointed out earlier, no. If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases (memory bottleneck excepted). It's not a processor designed to run Office or Firefox, but it may be very interesting on a server or as a competitor for the Xeon Phi. BTW, considering even my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
    - jallmann 12y ago
      > If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases Only if the tasks are trivially parallelizable. Any sequential dependencies among those tasks limit the effectiveness of parallelization. This is exactly what Amdahl's Law says. > my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors. Multiple browser tabs or open office documents are trivially parallelizable (eg, processes). The work within those tabs or documents is not linearly parallelizable -- running JS/macros, handling input, rendering, etc. Existing browser/office suites already do a pretty good job of splitting these tasks up, but they can't be split up arbitrarily into $core_count chunks.
      - rbanffy 12y ago
        > Only if the tasks are trivially parallelizable Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable. > Multiple browser tabs or open office documents are trivially parallelizable They would be if I could pay attention to more than one tab or document at any given time. In reality, I spend most of my time with full-screen apps and there is little point to keep other tabs or documents running.
        joosters 12y ago
        > Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable. Doesn't matter. You can (and probably will) still hit horrible bottlenecks in the network stack. Another flashback to developing on the 64+ Niagara cores. We were running an embarassingly scalable web application - loads of independent concurrent requests - but the Solaris OS just couldn't keep up. Has your NIC driver been optimised to scale for tens/hundreds of CPUs, I wonder?
    - zmanian 12y ago
      There is a lots of interesting work on building office suites and browsers that can take advantage of parallelism. But the general consensus is that we need to invent new lower level tools first. See Rust + Servo and MSFT + M#
  - higherpurpose 12y ago
    You're missing the point. I think everyone knows what you're saying is true for a mobile phone or even a desktop PC. Where it's NOT true is on servers, where tasks are "parallelized" since you have millions of concurrent users, so whether you have 1 million 8-core servers, or 125,000 64-core servers it doesn't matter much, because it's pretty much the same thing. But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores.
    - jallmann 12y ago
      This whole thread is talking past each other. Servers absolutely can have sequential dependencies, especially if there is a shared resource (eg, databases, IO, etc). If your application can make full use of all your cores without any inter-dependencies, great -- but that is not always the case.
    - ethbro 12y ago
      "But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores." How are you looking at this? From limited CompE background, my understanding was that moving data around core -> core burned a significant amount of power, even on the same die. (Using the comparison of one core vs two equally performing but separate cores)
- bitL 12y ago
  Actually, this holds only up to certain extent. Synchronization, locking, memory access patterns and context-switching from many threads with accompanied cache thrashing can bring the effective parallelization to very small levels. I remember from benchmarking a high performance transactional messaging system in Java I was working on seeing how it hit a flat line at some point where it didn't scale at all due to CPU/memory contention (this depended on maximal number of threads allowed). We had to look at alternative designs such as 0MQ to see how they overcame it - basically they put one working/client thread per core to achieve their 1M messages/s. We were using 10s of thousands of threads in our middleware when necessary, and that was faster than async NIO in Java 6 at that time.
rwmj 12y ago
I don't know why people are down-voting you. Amdahl's Law is absolutely applicable here. Having said that, I've been using a similar aarch64 machine for a few months (sorry, can't name it -- NDA), and single thread performance is just fine. I think people once used an RPi and now think every ARM has terrible performance. 64 bit ARM is a completely different architecture and far better performing.
- joosters 12y ago
  I'm a little biased through once having developed on Sun's Niagara processors. Early versions of these had pretty terrible single core (thread? I forget their exact terminology) performance. The theoretical power of the processor was huge but any time you hit a single-threaded program you felt it. ssh logins were slow, for example!
  - rwmj 12y ago
    I've been compiling libguestfs, libvirt, the kernel, qemu, and other programs, and using the aarch64 machine interactively, and it certainly doesn't feel slow. I have to be careful about exactly what I say here because the NDA prevents me from doing any kind of performance benchmarking (for good reasons: this is pre-release hardware and the performance doesn't reflect what the final hardware will be capable of), but ... performance is fine, I've had no problem with either single-threaded tasks or parallel compiles. Given I'm using it as a remote machine over ssh, it doesn't "feel" any different from an x86-64 server.
    - joosters 12y ago
      n.b. I wasn't very clear in my comment; my experience was on the original Niagara 1 & 2, this was about 7-8 years ago, I'd guess. Hopefully things have improved since :)
  - kjs3 12y ago
    It shouldn't be surprising that Niagra is slow on single threads. The whole point of the design was to support parallel, non-interactive tasks (e.g. databases) at the expense of non-threaded workloads. It was a design trade-off that Sun was very up front about. Even N1 & N2 gave remarkable price/performance numbers when used for the tasks it was designed for.
fulafel 12y ago
For the optimist, Amdahl's law is just a restatement of Zeno's halfway paradox ;)
colanderman 12y ago
But if you have only 10 processors, you'll only get a 5× speedup. 36 processors will get you an 8× speedup. (After that point it's really diminishing returns; you'd need 81 processors to get a 9× speedup.)
deleted 12y ago
[deleted]
- joosters 12y ago
  ...counting OS overhead, drive I/O, loading, cache misses Let's hope these unpleasantly difficult bits of the OS and hardware also scale well. If they don't all make perfect use of your 300 cores then your userland code won't, either.

xienze 12y ago

Well, you're thinking in terms of parallelising a single task. For a server, you're talking about a large number of concurrent tasks (users), so more cores are better in that case.

joosters 12y ago
That's basically a truism. If you have parallelisable tasks, then they will parallelise. No surprise there. I was trying to make the point that as we step up the number of cores, the actual gain is less and less.
- deleted 12y ago
  [deleted]
- rbanffy 12y ago
  Again, as it was pointed out earlier, no. If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases (memory bottleneck excepted). It's not a processor designed to run Office or Firefox, but it may be very interesting on a server or as a competitor for the Xeon Phi. BTW, considering even my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors.
  - jallmann 12y ago
    > If you have 48 runnable threads from 48 different tasks running on your machine, you'll get pretty much linear performance increases Only if the tasks are trivially parallelizable. Any sequential dependencies among those tasks limit the effectiveness of parallelization. This is exactly what Amdahl's Law says. > my humble laptop can do 8 threads at once, it's about time we started building office suites and browsers to take advantage of these processors. Multiple browser tabs or open office documents are trivially parallelizable (eg, processes). The work within those tabs or documents is not linearly parallelizable -- running JS/macros, handling input, rendering, etc. Existing browser/office suites already do a pretty good job of splitting these tasks up, but they can't be split up arbitrarily into $core_count chunks.
    - rbanffy 12y ago
      > Only if the tasks are trivially parallelizable Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable. > Multiple browser tabs or open office documents are trivially parallelizable They would be if I could pay attention to more than one tab or document at any given time. In reality, I spend most of my time with full-screen apps and there is little point to keep other tabs or documents running.
      - joosters 12y ago
        > Which is a very common scenario on web and application servers, specially if you design your web applications to be scalable. Doesn't matter. You can (and probably will) still hit horrible bottlenecks in the network stack. Another flashback to developing on the 64+ Niagara cores. We were running an embarassingly scalable web application - loads of independent concurrent requests - but the Solaris OS just couldn't keep up. Has your NIC driver been optimised to scale for tens/hundreds of CPUs, I wonder?
  - zmanian 12y ago
    There is a lots of interesting work on building office suites and browsers that can take advantage of parallelism. But the general consensus is that we need to invent new lower level tools first. See Rust + Servo and MSFT + M#
- higherpurpose 12y ago
  You're missing the point. I think everyone knows what you're saying is true for a mobile phone or even a desktop PC. Where it's NOT true is on servers, where tasks are "parallelized" since you have millions of concurrent users, so whether you have 1 million 8-core servers, or 125,000 64-core servers it doesn't matter much, because it's pretty much the same thing. But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores.
  - jallmann 12y ago
    This whole thread is talking past each other. Servers absolutely can have sequential dependencies, especially if there is a shared resource (eg, databases, IO, etc). If your application can make full use of all your cores without any inter-dependencies, great -- but that is not always the case.
  - ethbro 12y ago
    "But from a whole datacenter perspective, I believe having servers with more cores is more efficient than having more servers with fewer cores, since data travels faster between more of the cores." How are you looking at this? From limited CompE background, my understanding was that moving data around core -> core burned a significant amount of power, even on the same die. (Using the comparison of one core vs two equally performing but separate cores)
bitL 12y ago
Actually, this holds only up to certain extent. Synchronization, locking, memory access patterns and context-switching from many threads with accompanied cache thrashing can bring the effective parallelization to very small levels. I remember from benchmarking a high performance transactional messaging system in Java I was working on seeing how it hit a flat line at some point where it didn't scale at all due to CPU/memory contention (this depended on maximal number of threads allowed). We had to look at alternative designs such as 0MQ to see how they overcame it - basically they put one working/client thread per core to achieve their 1M messages/s. We were using 10s of thousands of threads in our middleware when necessary, and that was faster than async NIO in Java 6 at that time.

rwmj 12y ago

I don't know why people are down-voting you. Amdahl's Law is absolutely applicable here. Having said that, I've been using a similar aarch64 machine for a few months (sorry, can't name it -- NDA), and single thread performance is just fine. I think people once used an RPi and now think every ARM has terrible performance. 64 bit ARM is a completely different architecture and far better performing.

joosters 12y ago
I'm a little biased through once having developed on Sun's Niagara processors. Early versions of these had pretty terrible single core (thread? I forget their exact terminology) performance. The theoretical power of the processor was huge but any time you hit a single-threaded program you felt it. ssh logins were slow, for example!
- rwmj 12y ago
  I've been compiling libguestfs, libvirt, the kernel, qemu, and other programs, and using the aarch64 machine interactively, and it certainly doesn't feel slow. I have to be careful about exactly what I say here because the NDA prevents me from doing any kind of performance benchmarking (for good reasons: this is pre-release hardware and the performance doesn't reflect what the final hardware will be capable of), but ... performance is fine, I've had no problem with either single-threaded tasks or parallel compiles. Given I'm using it as a remote machine over ssh, it doesn't "feel" any different from an x86-64 server.
  - joosters 12y ago
    n.b. I wasn't very clear in my comment; my experience was on the original Niagara 1 & 2, this was about 7-8 years ago, I'd guess. Hopefully things have improved since :)
- kjs3 12y ago
  It shouldn't be surprising that Niagra is slow on single threads. The whole point of the design was to support parallel, non-interactive tasks (e.g. databases) at the expense of non-threaded workloads. It was a design trade-off that Sun was very up front about. Even N1 & N2 gave remarkable price/performance numbers when used for the tasks it was designed for.

fulafel 12y ago

For the optimist, Amdahl's law is just a restatement of Zeno's halfway paradox ;)

colanderman 12y ago

But if you have only 10 processors, you'll only get a 5× speedup. 36 processors will get you an 8× speedup. (After that point it's really diminishing returns; you'd need 81 processors to get a 9× speedup.)

deleted 12y ago

[deleted]

joosters 12y ago
...counting OS overhead, drive I/O, loading, cache misses Let's hope these unpleasantly difficult bits of the OS and hardware also scale well. If they don't all make perfect use of your 300 cores then your userland code won't, either.