5 ms·

>That you can program GPUs in a C/C++-like language does not entail that C/C++ is a natural form of expression for that architecture. It's not a matter of what

>That you can program GPUs in a C/C++-like language does not entail that C/C++ is a natural form of expression for that architecture. It's not a matter of what is/isn't a "natural form of expression." The point of C/C++ is to be high-level enough for humans to build their own abstractions over hardware. (sounds like an OS, right?) The success of the design of C/C++ is in that the creators had no knowledge of modern GPUs, yet GPUs can efficiently execute them with a little care from developers. We use other abstractions (e.g. SciPy on Tensorflow) because they are more appropriate to solve our problems, but they are built on C. >Lisp machines and tagged hardware architectures do make such a distinction though, and so more naturally fit. With such distinctions, you could even have a hardware GC. And why would that not be backwards-compatible with legacy C? Particularly, I am rejecting the idea that C is somehow stunting hardware development - I see no evidence of this fact. I am also skeptical about the claim (although I will not reject it outright) that there is a language substantially better fit compared to C for low-level programming (e.g. embedded, kernel).

naasking 7y ago
> It's not a matter of what is/isn't a "natural form of expression." The point of C/C++ is to be high-level enough for humans to build their own abstractions over hardware. Sure it matters. If primitives don't map naturally to the hardware, then you have to build a runtime to emulate those primitives, just like GC'd languages do. > The success of the design of C/C++ is in that the creators had no knowledge of modern GPUs, yet GPUs can efficiently execute them with a little care from developers You cannot run any arbitrary C program on a GPU. This fact is exactly why GPUs were able to innovate without legacy compatibility holding them back. Only later were GPUs generalised to support more sophisticated programs, which then permitted a subset of C to execute efficiently. The progress of GPUs proves exactly the opposite point that you are claiming. If C were so perfectly suited to any sort of hardware, then GPUs would have been able to run C programs right from the beginning, which is not true. > And why would that not be backwards-compatible with legacy C? That's not the point I'm making. Turing equivalence ensures that compatibility can be assured no matter what. The actual point is that CPU innovations were tested against C benchmark suites to check whether innovations effectively improved performance, and some or many of those that failed to show meaningful improvements were discarded, despite the fact that they would have had other benefits (obviously not all of them, but enough). It's simply natural selection for CPU innovation. It's incredibly naive to think that only hardware influences software and not vice versa. For instance, who would create a hardware architecture that didn't have pointers? It would simply never happen, because efficient C compatibility is too important. The problem is that C was given a disproportionately heavy weighting in these decisions. For instance, a tagged memory architecture would show zero improvement on C benchmarks, but it would have been huge for the languages that now dominate the software industry. > that there is a language substantially better fit compared to C for low-level programming (e.g. embedded, kernel). The limitations of C are well known (poor bit fields and bit manipulation, poor support for alignment and padding, no modules, poor standard library, etc, etc.). Zig addresses some of those issues. Ada has been better than C for a long time. A better language than all of these could definitely be designed given enough resources, eg. see the research effort "House" [1]. [1] http://programatica.cs.pdx.edu/House/ http://programatica.cs.pdx.edu/House/
- madmax96 7y ago
  >If primitives don't map naturally to the hardware, then you have to build a runtime to emulate those primitives, just like GC'd languages do. That's only half the equation. Hardware cannot save you from semantics that are less efficient. To use your example: every GC'd language must have a runtime system track objects, whether that is implemented with or without hardware support. That system constitutes additional overhead -- either precious silicon is used delivering hardware support for GC or clock cycles are used emulating that support. Either way, you're losing performance. C/C++ have semantics that are easy to support, in contrast. >You cannot run any arbitrary C program on a GPU. Nor can you run any arbitrary C/C++ program written for Posix on Windows, or a program written for the x86 on a STM32, etc. You have always had to know your platform with C/C++. The point is that they are flexible enough to work very well on many platforms. >This fact is exactly why GPUs were able to innovate without legacy compatibility holding them back. GPUs have become a lucrative business precisely because they have begun exposing a C++ interface. Look at how the usage of graphics cards have changed in recenter years. > If C were so perfectly suited to any sort of hardware, then GPUs would have been able to run C programs right from the beginning, which is not true. No. GPUs _were not_ general purpose compute devices from the beginning, as you pointed out. You had GLSL, etc. but the interface exposed to programmers was not Turing-complete. From what I gather, GPUs have only had a Turing-complete interface since shader model 3.0, which first appeared in 2004. By 2007, you had nvcc. Today, C++ is very well supported by CUDA. You may as well be saying "You can't run C on a cardboard box, so it's obviously not well-suited to all hardware." Obviously, your hardware needs to expose a Turing-complete interface for a Turing-complete language to be able to run on it. >The problem is that C was given a disproportionately heavy weighting in these decisions. For instance, a tagged memory architecture would show zero improvement on C benchmarks, but it would have been huge for the languages that now dominate the software industry. At what cost? As I already pointed out, adding support for VHLLs at the hardware level means you are spending silicon space on that task => languages like C will be slower. Yes, a lot of software is written in JavaScript, Java, and Python, and these languages would benefit from that hardware support. But people using JavaScript, Java, and Python generally are relying on C services (memcached, redis, postgre, etc) to do their heavy-lifting anyway, which you just made slower. >For instance, who would create a hardware architecture that didn't have pointers? It would simply never happen, because efficient C compatibility is too important. No. It would never happen because the machine you just described would make a very bad general purpose computer. >The limitations of C are well known Yes, they are. But everything you listed isn't substantial. It's C, with a better standard library, standard support for controlling alignment/padding, and modules. That's not significantly different.

naasking 7y ago

> It's not a matter of what is/isn't a "natural form of expression." The point of C/C++ is to be high-level enough for humans to build their own abstractions over hardware. Sure it matters. If primitives don't map naturally to the hardware, then you have to build a runtime to emulate those primitives, just like GC'd languages do. > The success of the design of C/C++ is in that the creators had no knowledge of modern GPUs, yet GPUs can efficiently execute them with a little care from developers You cannot run any arbitrary C program on a GPU. This fact is exactly why GPUs were able to innovate without legacy compatibility holding them back. Only later were GPUs generalised to support more sophisticated programs, which then permitted a subset of C to execute efficiently. The progress of GPUs proves exactly the opposite point that you are claiming. If C were so perfectly suited to any sort of hardware, then GPUs would have been able to run C programs right from the beginning, which is not true. > And why would that not be backwards-compatible with legacy C? That's not the point I'm making. Turing equivalence ensures that compatibility can be assured no matter what. The actual point is that CPU innovations were tested against C benchmark suites to check whether innovations effectively improved performance, and some or many of those that failed to show meaningful improvements were discarded, despite the fact that they would have had other benefits (obviously not all of them, but enough). It's simply natural selection for CPU innovation. It's incredibly naive to think that only hardware influences software and not vice versa. For instance, who would create a hardware architecture that didn't have pointers? It would simply never happen, because efficient C compatibility is too important. The problem is that C was given a disproportionately heavy weighting in these decisions. For instance, a tagged memory architecture would show zero improvement on C benchmarks, but it would have been huge for the languages that now dominate the software industry. > that there is a language substantially better fit compared to C for low-level programming (e.g. embedded, kernel). The limitations of C are well known (poor bit fields and bit manipulation, poor support for alignment and padding, no modules, poor standard library, etc, etc.). Zig addresses some of those issues. Ada has been better than C for a long time. A better language than all of these could definitely be designed given enough resources, eg. see the research effort "House" [1]. [1] http://programatica.cs.pdx.edu/House/ http://programatica.cs.pdx.edu/House/

madmax96 7y ago
>If primitives don't map naturally to the hardware, then you have to build a runtime to emulate those primitives, just like GC'd languages do. That's only half the equation. Hardware cannot save you from semantics that are less efficient. To use your example: every GC'd language must have a runtime system track objects, whether that is implemented with or without hardware support. That system constitutes additional overhead -- either precious silicon is used delivering hardware support for GC or clock cycles are used emulating that support. Either way, you're losing performance. C/C++ have semantics that are easy to support, in contrast. >You cannot run any arbitrary C program on a GPU. Nor can you run any arbitrary C/C++ program written for Posix on Windows, or a program written for the x86 on a STM32, etc. You have always had to know your platform with C/C++. The point is that they are flexible enough to work very well on many platforms. >This fact is exactly why GPUs were able to innovate without legacy compatibility holding them back. GPUs have become a lucrative business precisely because they have begun exposing a C++ interface. Look at how the usage of graphics cards have changed in recenter years. > If C were so perfectly suited to any sort of hardware, then GPUs would have been able to run C programs right from the beginning, which is not true. No. GPUs _were not_ general purpose compute devices from the beginning, as you pointed out. You had GLSL, etc. but the interface exposed to programmers was not Turing-complete. From what I gather, GPUs have only had a Turing-complete interface since shader model 3.0, which first appeared in 2004. By 2007, you had nvcc. Today, C++ is very well supported by CUDA. You may as well be saying "You can't run C on a cardboard box, so it's obviously not well-suited to all hardware." Obviously, your hardware needs to expose a Turing-complete interface for a Turing-complete language to be able to run on it. >The problem is that C was given a disproportionately heavy weighting in these decisions. For instance, a tagged memory architecture would show zero improvement on C benchmarks, but it would have been huge for the languages that now dominate the software industry. At what cost? As I already pointed out, adding support for VHLLs at the hardware level means you are spending silicon space on that task => languages like C will be slower. Yes, a lot of software is written in JavaScript, Java, and Python, and these languages would benefit from that hardware support. But people using JavaScript, Java, and Python generally are relying on C services (memcached, redis, postgre, etc) to do their heavy-lifting anyway, which you just made slower. >For instance, who would create a hardware architecture that didn't have pointers? It would simply never happen, because efficient C compatibility is too important. No. It would never happen because the machine you just described would make a very bad general purpose computer. >The limitations of C are well known Yes, they are. But everything you listed isn't substantial. It's C, with a better standard library, standard support for controlling alignment/padding, and modules. That's not significantly different.