7 ms·

TLDR: C was close-to-the-metal on the PDP-11 but since then hardware has become more complex while exposing the same abstraction to the C programmer. That means

TLDR: C was close-to-the-metal on the PDP-11 but since then hardware has become more complex while exposing the same abstraction to the C programmer. That means that hardware features such as speculative execution and L1/L2 caching are invisible to the programmer. This was the cause of Spectre and Meltdown and it forces a lot of complexity into the compiler. GPUs achieve high performance in part because their programming model goes beyond C. Processors would be able to evolve if they weren't hamstrung by having to support C.

arghwhat 8y ago
It is very important to emphasize that GPU's only "achieve high performance" in workloads tailored very specifically to their extremely limited architecture. CPU's, on the other hand, are designed to be much more generic with decent performance for any task.
- derefr 8y ago
  I wouldn't say this is precisely true. Look at Dolphin's "ubershaders" (https://dolphin-emu.org/blog/2017/07/30/ubershaders/ https://dolphin-emu.org/blog/2017/07/30/ubershaders/): they're essentially a Turing machine, running on your GPU, used to emulate a GPU of another architecture... and yet this is still (much!) faster than doing the same on the CPU. And there's nothing special about emulating a GPU on a GPU; you could emulate a CPU architecture just as easily, at a much higher level than you get from an FPGA, and so perhaps faster than you'd be able to get from today's FPGAs. And, if you're mapping GPU shader units 1:1 to VM schedulers, you'd also get a far higher degree of core parallelism than even a Xeon Phi-like architecture would give you. (The big limitation is that you'd be very limited in I/O bandwidth out to main memory; but each shader unit would be able to reserve its own small amount of VRAM texture space—i.e. NUMA memory—to work with.) I'm still waiting for someone to port Erlang's BEAM VM to run on a GPU; it'd be a perfect fit. :)
- umanwizard 8y ago
  I think it's more accurate to say that CPUs are high-performance for different tasks than GPUs. Simple code, very wide data workloads are atrociously slow on CPUs, and single-threaded heavily branching workloads are atrociously slow on GPUs. That doesn't mean that one is more limited than the other.
  - arghwhat 8y ago
    I would argue that the total set of "good performance" workloads are smaller on a GPU than they are on a CPU. However, "good performance" on a CPU is much much worse than "good performance" on a GPU. CPU's just achieve their mediocre performance on a larger set of usecases. GPU's are specialized devices that are very good at specialized activities.
  - hexane360 8y ago
    The whole point of the article is that this is largely a myth imposed by the memory model of C and C-like languages. Single-threaded heavily branching workloads would be atrociously slow on modern CPUs, if not for branch prediction (which caused Spectre). On modern processors, you have 180 instructions running in one thread. The processor is doing an ok job filling these instructions, but a language and compiler can do a much better job. C doesn't collect any information about data dependencies, and instead just pretends all instructions are sequential. Even code which contains tight loops of sequential commands can be optimized, because you have an entire program and operating system running around that sequential code.
occamrazor 8y ago
I don’t understand. CPUs do not support C, they support a specific instruction set. What stops them from having instructions for cache management, pipelining, speculative execution hints, etc?
- tathougies 8y ago
  The Itanium processor did exactly this. Other than being a commercial flop, it was found to be quite difficult to actually get the compiler to generate good management instructions, and x86 was often able to beat out an itanium core at the same clock speed
  - hexane360 8y ago
    It seems like x86 exists at a local maxima, and Itanium didn't go far enough away to find a different peak.
- coliveira 8y ago
  They do not support C officially, but every CPU designer knows that 99%+ of the code that matters is written in C. Therefore they design chips targeting this translation from C. What the authors want is a better lower level interface that would allow for modern processor features without the legacy of the features available to the PDP11.
  - umanwizard 8y ago
    > 99%+ of the code that matters is written in C I think a better way of stating this is "99% of the code that matters is written in C, or in a language designed with a similar target architecture as C in mind". Certainly a lot of code that matters is written in C++, Objective-C, and Java, but the same points hold true for all of those.
- mattnewport 8y ago
  Many of these features are very difficult to use in a useful way in a static context (e.g. at compile time) because the performance gains mostly come from taking advantage of dynamic context. Speculative execution and out of order execution for example are mostly useful because you don't know at compile time exactly what data your code is processing or what CPU it is running on, what function / context you are being called from, what is in cache and what isn't, etc. The SPUs on the PlayStation 3 were an experiment in user managed caches and that proved to be a difficult thing to make effective use of even in games where you know more context than a lot of code can assume.
jackhack 8y ago
Thanks for the TLDR. But if that's the argument, then not even assembly is sufficient, as control over speculative branching and prefetch is only accessible via microcode in the CPU. I think the argument is improperly framed. This is a discussion over public and private interface. The CPU is treated as a black box with a public interface (the x86+ instruction set). Precisely how those instructions are implemented (on chip microcode) is a private matter for the chip design team, which if correctly implemented, does not matter to the user, as the results should be correct and consistent. Obviously, a poor implementation can lead to Spectre or Meltdown. But for the most part the specific transistors & diodes used to sum a set of integers, or transfer a word from L2 to L3 cache, etc. shouldn't matter to us. If the compilers are relying on side effects to alter behavior of the internal implementation based on performance evidence, then that is a boundary violation. C is low level. It remains "universal assembly language".
- umanwizard 8y ago
  You make good points - if we're just talking about semantics, then yes C is the closest portable language to x86 or arm and is low-level in that sense. But on the other hand, semantics is not always the only important thing: performance is sometimes important also, and there these low-level details matter. The architecture does its best to hide them from the user, but the abstraction is very leaky. For example, when writing high-performance CPU-bound code it's usually important to keep in mind how wide cache lines are, but C doesn't expose this to the programmer in a natural way.
- PeterisP 8y ago
  The argument implied in the article is that choosing a different public interface (breaking "C compatibility" and the imposted limitations) could bring a serious performance improvement. While precisely how those instructions are implemented (on chip microcode) is a private matter for the chip design team, we do care how much resources it takes to implement these instructions, since if we can enable a more efficient implementation then we can get better price/performance.
  - kartickv 8y ago
kevstev 8y ago
I was with you until the last sentence: "Processors would be able to evolve if they weren't hamstrung by having to support C." I don't think its fair or correct to say that C is the real issue. Recently there have been languages like erlang and support for more functional models that make concurrent code a lot easier to write. The first real consumer multicore processors were only released a bit over 10 years with Intel's Core 2 duo's. Of course SMP systems existed before that, Sun had them for years, but they were relatively niche. Still, Java, C++, C#, are all languages that produce much easier to maintain code if they are single threaded. Recent darlings like JS and Python are single threaded out of the box. The large majority of languages in use today are not designed to be concurrent as a first principle. True multicore systems have been around for decades, software and mindshare is now starting to catch up and use tools that make concurrency easy.
- jacquesm 8y ago
  > Recently there have been languages like erlang Erlang is decades old. It's 32, only 16 years younger than C.
- SomeHacker44 8y ago
  My Symbolics computers (running Symbolics Ivory processors) run Lisp really well, as well as C - they have a C compiler. I have operational computers of a variety of architectures at home, including the oldest generations (6502, 680x0), Sparc, Symbolics, DEC Alpha, MIPS 32- and 64-bit, etc., and even an extremely rare (and unfortunately not-running) Multiflow, the granddaddy of VLIW. My favorite part of the original article was the final section. I wish we had a modern CPU renassiance akin to what was going on in the 80s and 90s, but the market dominance of x64 and ARM seems to be squelching things, with optimizations to those architectures rather than novel new ones (with possibly novel new compiler technologies). 64-bit ARM was a nice little improvement, though.
ajross 8y ago
Yeah, but that's just reinventing the mistakes of VLIW all over again. Yes, CPUs have complicated behavior in a way that can't be captured by scalar imperative languages in a concise way. No, that doesn't mean that you can fix this with new abstractions. The reason C won wasn't that it forced CPUs to adhere to its particular execution metaphor[1], but that it happened upon a metaphor that could be easily expressed and supported by CPUs as they evolved over decades of progress. [1] Basically: byte-addressable memory in a single linear space, a high performance grows-down stack in that same memory space, two's complement arithmetic, and "unsurprising" cache coherence behavior. No, the last three aren't technically part of the language spec, but they're part of the model nonetheless and had successful architectures really diverged there I doubt C-like runtimes would have "won".

arghwhat 8y ago

It is very important to emphasize that GPU's only "achieve high performance" in workloads tailored very specifically to their extremely limited architecture. CPU's, on the other hand, are designed to be much more generic with decent performance for any task.

derefr 8y ago
I wouldn't say this is precisely true. Look at Dolphin's "ubershaders" (https://dolphin-emu.org/blog/2017/07/30/ubershaders/ https://dolphin-emu.org/blog/2017/07/30/ubershaders/): they're essentially a Turing machine, running on your GPU, used to emulate a GPU of another architecture... and yet this is still (much!) faster than doing the same on the CPU. And there's nothing special about emulating a GPU on a GPU; you could emulate a CPU architecture just as easily, at a much higher level than you get from an FPGA, and so perhaps faster than you'd be able to get from today's FPGAs. And, if you're mapping GPU shader units 1:1 to VM schedulers, you'd also get a far higher degree of core parallelism than even a Xeon Phi-like architecture would give you. (The big limitation is that you'd be very limited in I/O bandwidth out to main memory; but each shader unit would be able to reserve its own small amount of VRAM texture space—i.e. NUMA memory—to work with.) I'm still waiting for someone to port Erlang's BEAM VM to run on a GPU; it'd be a perfect fit. :)
umanwizard 8y ago
I think it's more accurate to say that CPUs are high-performance for different tasks than GPUs. Simple code, very wide data workloads are atrociously slow on CPUs, and single-threaded heavily branching workloads are atrociously slow on GPUs. That doesn't mean that one is more limited than the other.
- arghwhat 8y ago
  I would argue that the total set of "good performance" workloads are smaller on a GPU than they are on a CPU. However, "good performance" on a CPU is much much worse than "good performance" on a GPU. CPU's just achieve their mediocre performance on a larger set of usecases. GPU's are specialized devices that are very good at specialized activities.
- hexane360 8y ago
  The whole point of the article is that this is largely a myth imposed by the memory model of C and C-like languages. Single-threaded heavily branching workloads would be atrociously slow on modern CPUs, if not for branch prediction (which caused Spectre). On modern processors, you have 180 instructions running in one thread. The processor is doing an ok job filling these instructions, but a language and compiler can do a much better job. C doesn't collect any information about data dependencies, and instead just pretends all instructions are sequential. Even code which contains tight loops of sequential commands can be optimized, because you have an entire program and operating system running around that sequential code.

occamrazor 8y ago

I don’t understand. CPUs do not support C, they support a specific instruction set. What stops them from having instructions for cache management, pipelining, speculative execution hints, etc?

tathougies 8y ago
The Itanium processor did exactly this. Other than being a commercial flop, it was found to be quite difficult to actually get the compiler to generate good management instructions, and x86 was often able to beat out an itanium core at the same clock speed
- hexane360 8y ago
  It seems like x86 exists at a local maxima, and Itanium didn't go far enough away to find a different peak.
coliveira 8y ago
They do not support C officially, but every CPU designer knows that 99%+ of the code that matters is written in C. Therefore they design chips targeting this translation from C. What the authors want is a better lower level interface that would allow for modern processor features without the legacy of the features available to the PDP11.
- umanwizard 8y ago
  > 99%+ of the code that matters is written in C I think a better way of stating this is "99% of the code that matters is written in C, or in a language designed with a similar target architecture as C in mind". Certainly a lot of code that matters is written in C++, Objective-C, and Java, but the same points hold true for all of those.
mattnewport 8y ago
Many of these features are very difficult to use in a useful way in a static context (e.g. at compile time) because the performance gains mostly come from taking advantage of dynamic context. Speculative execution and out of order execution for example are mostly useful because you don't know at compile time exactly what data your code is processing or what CPU it is running on, what function / context you are being called from, what is in cache and what isn't, etc. The SPUs on the PlayStation 3 were an experiment in user managed caches and that proved to be a difficult thing to make effective use of even in games where you know more context than a lot of code can assume.

jackhack 8y ago

Thanks for the TLDR. But if that's the argument, then not even assembly is sufficient, as control over speculative branching and prefetch is only accessible via microcode in the CPU. I think the argument is improperly framed. This is a discussion over public and private interface. The CPU is treated as a black box with a public interface (the x86+ instruction set). Precisely how those instructions are implemented (on chip microcode) is a private matter for the chip design team, which if correctly implemented, does not matter to the user, as the results should be correct and consistent. Obviously, a poor implementation can lead to Spectre or Meltdown. But for the most part the specific transistors & diodes used to sum a set of integers, or transfer a word from L2 to L3 cache, etc. shouldn't matter to us. If the compilers are relying on side effects to alter behavior of the internal implementation based on performance evidence, then that is a boundary violation. C is low level. It remains "universal assembly language".

umanwizard 8y ago
You make good points - if we're just talking about semantics, then yes C is the closest portable language to x86 or arm and is low-level in that sense. But on the other hand, semantics is not always the only important thing: performance is sometimes important also, and there these low-level details matter. The architecture does its best to hide them from the user, but the abstraction is very leaky. For example, when writing high-performance CPU-bound code it's usually important to keep in mind how wide cache lines are, but C doesn't expose this to the programmer in a natural way.
PeterisP 8y ago
The argument implied in the article is that choosing a different public interface (breaking "C compatibility" and the imposted limitations) could bring a serious performance improvement. While precisely how those instructions are implemented (on chip microcode) is a private matter for the chip design team, we do care how much resources it takes to implement these instructions, since if we can enable a more efficient implementation then we can get better price/performance.
- kartickv 8y ago

kevstev 8y ago

I was with you until the last sentence: "Processors would be able to evolve if they weren't hamstrung by having to support C." I don't think its fair or correct to say that C is the real issue. Recently there have been languages like erlang and support for more functional models that make concurrent code a lot easier to write. The first real consumer multicore processors were only released a bit over 10 years with Intel's Core 2 duo's. Of course SMP systems existed before that, Sun had them for years, but they were relatively niche. Still, Java, C++, C#, are all languages that produce much easier to maintain code if they are single threaded. Recent darlings like JS and Python are single threaded out of the box. The large majority of languages in use today are not designed to be concurrent as a first principle. True multicore systems have been around for decades, software and mindshare is now starting to catch up and use tools that make concurrency easy.

jacquesm 8y ago
> Recently there have been languages like erlang Erlang is decades old. It's 32, only 16 years younger than C.
SomeHacker44 8y ago
My Symbolics computers (running Symbolics Ivory processors) run Lisp really well, as well as C - they have a C compiler. I have operational computers of a variety of architectures at home, including the oldest generations (6502, 680x0), Sparc, Symbolics, DEC Alpha, MIPS 32- and 64-bit, etc., and even an extremely rare (and unfortunately not-running) Multiflow, the granddaddy of VLIW. My favorite part of the original article was the final section. I wish we had a modern CPU renassiance akin to what was going on in the 80s and 90s, but the market dominance of x64 and ARM seems to be squelching things, with optimizations to those architectures rather than novel new ones (with possibly novel new compiler technologies). 64-bit ARM was a nice little improvement, though.

ajross 8y ago

Yeah, but that's just reinventing the mistakes of VLIW all over again. Yes, CPUs have complicated behavior in a way that can't be captured by scalar imperative languages in a concise way. No, that doesn't mean that you can fix this with new abstractions. The reason C won wasn't that it forced CPUs to adhere to its particular execution metaphor[1], but that it happened upon a metaphor that could be easily expressed and supported by CPUs as they evolved over decades of progress. [1] Basically: byte-addressable memory in a single linear space, a high performance grows-down stack in that same memory space, two's complement arithmetic, and "unsurprising" cache coherence behavior. No, the last three aren't technically part of the language spec, but they're part of the model nonetheless and had successful architectures really diverged there I doubt C-like runtimes would have "won".