5 ms·

This problem has been solved - albeit sub-optimally for general purpose computing. GPUs are massively parallel processors with 1000s of cores. While CUDA isn’t

This problem has been solved - albeit sub-optimally for general purpose computing. GPUs are massively parallel processors with 1000s of cores. While CUDA isn’t the easiest to work with, some Python libraries such as JAX and Tai-chi are attemting to remove the bar between CPU and GPU computing completely. A program using either cab transparently switch between a CPU or GPU backend.

timlatim 4y ago
GPUs do not have thousands of physical cores, though. What's called a "CUDA core" is essentially a programming slash marketing abstraction for a SIMD lane. A closer analogy to a CPU core would be a Streaming Multiprocessor (RTX 4090, for instance, has 128 of them). But that comparison is still moot, because GPUs are simply not designed for executing branchy scalar code. They'd be laughably slow at it, so outside of easily vectorizable code that already takes advantage of SIMD instructions, I don't see how you could offload any CPU tasks to a GPU.
- blagie 4y ago
  I think the major architectural leap would be MIMD with a standard ISA. I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X. I'd take that over my GPU any day. The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.
  - jacquesm 4y ago
    I'd love to see solutions like that. But what you are talking about is probably a niche market.
    - blagie 4y ago
      I don't think so. I think the core problem are network effects and momentum. If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight. Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document. However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.
      - jacquesm 4y ago
        I think the big driver could be security.
  - lizknope 4y ago
    https://en.wikipedia.org/wiki/Multiple_instruction,_multiple_data https://en.wikipedia.org/wiki/Multiple_instruction,_multiple... An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture. https://en.wikipedia.org/wiki/Larrabee_(microarchitecture) https://en.wikipedia.org/wiki/Larrabee_(microarchitecture) Its x86 cores were based on the much simpler P54C Pentium https://en.wikipedia.org/wiki/Xeon_Phi https://en.wikipedia.org/wiki/Xeon_Phi That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.
    - jacquesm 4y ago
      I tried buying those but they wouldn't let me.
      - monocasa 4y ago
        Well, FWIW xeon phis are about $40 on ebay now. Theyre obviously not much more than toy's now.
jacquesm 4y ago
While there is a trend towards GPGPU this isn't there at the level where you can just go out and buy it like the dev kit in TFA. GPU compute capacity is still limited to a subset of all the use cases that you can use an ordinary CPU for and even though there is a chance that in the future the two will converge for now they are far enough from each other that I don't see the gain in the same sense that a large number of 'regular' cores would. The fact that programs can switch between those two execution contexts seamlessly is a nice benefit but still isn't the same as having many ordinary CPUs. I've used GPUs extensively and I'm really happy to see their capabilities more integrated in everyday languages but they are at best for now a co-processor like device, a chunk of hardware that you offload specific parts of your workload (typically: the numerical chunk of it that is massively parallel).
crabbone 4y ago
This is a shot in the wrong direction. GPUs are accelerators for code that, ultimately is run by CPUs. They might be headed to be more all-purpose computing hardware, but they aren't there yet, most importantly, from OS perspective. To elaborate on this: OS creates processes, assigns ids to them, assigns other physical resources to them s.a. association with namespaces (which later gives them user permissions, network access, virtual memory access, filesystem access etc.) and then these processes are associated with some GPU resource. If and when OS will start creating processes entirely on GPU, then it will make sense to talk about how GPUs are solving the problems of threads per core etc. For now it's a moot point. This is not said to discourage though. I really feel like CPU-centric model of what we call "computers" is not a good one going forward. The "periphery" is growing smarter with each generation, and wants to do its own computing, and spread its load somehow, and we keep coming up with ad hoc solutions that don't mix well with CPU-based concurrency, s.a. async I/O or CUDA. We really need a different concept of concurrency that would be more uniform and at the same time more flexible across different devices that can do work concurrently, and this, interface if you will, must come from the operating system, not as a user-space library to be truly effective.

timlatim 4y ago

GPUs do not have thousands of physical cores, though. What's called a "CUDA core" is essentially a programming slash marketing abstraction for a SIMD lane. A closer analogy to a CPU core would be a Streaming Multiprocessor (RTX 4090, for instance, has 128 of them). But that comparison is still moot, because GPUs are simply not designed for executing branchy scalar code. They'd be laughably slow at it, so outside of easily vectorizable code that already takes advantage of SIMD instructions, I don't see how you could offload any CPU tasks to a GPU.

blagie 4y ago
I think the major architectural leap would be MIMD with a standard ISA. I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X. I'd take that over my GPU any day. The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.
- jacquesm 4y ago
  I'd love to see solutions like that. But what you are talking about is probably a niche market.
  - blagie 4y ago
    I don't think so. I think the core problem are network effects and momentum. If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight. Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document. However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.
    - jacquesm 4y ago
      I think the big driver could be security.
- lizknope 4y ago
  https://en.wikipedia.org/wiki/Multiple_instruction,_multiple_data https://en.wikipedia.org/wiki/Multiple_instruction,_multiple... An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture. https://en.wikipedia.org/wiki/Larrabee_(microarchitecture) https://en.wikipedia.org/wiki/Larrabee_(microarchitecture) Its x86 cores were based on the much simpler P54C Pentium https://en.wikipedia.org/wiki/Xeon_Phi https://en.wikipedia.org/wiki/Xeon_Phi That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.
  - jacquesm 4y ago
    I tried buying those but they wouldn't let me.
    - monocasa 4y ago
      Well, FWIW xeon phis are about $40 on ebay now. Theyre obviously not much more than toy's now.

jacquesm 4y ago

While there is a trend towards GPGPU this isn't there at the level where you can just go out and buy it like the dev kit in TFA. GPU compute capacity is still limited to a subset of all the use cases that you can use an ordinary CPU for and even though there is a chance that in the future the two will converge for now they are far enough from each other that I don't see the gain in the same sense that a large number of 'regular' cores would. The fact that programs can switch between those two execution contexts seamlessly is a nice benefit but still isn't the same as having many ordinary CPUs. I've used GPUs extensively and I'm really happy to see their capabilities more integrated in everyday languages but they are at best for now a co-processor like device, a chunk of hardware that you offload specific parts of your workload (typically: the numerical chunk of it that is massively parallel).

crabbone 4y ago

This is a shot in the wrong direction. GPUs are accelerators for code that, ultimately is run by CPUs. They might be headed to be more all-purpose computing hardware, but they aren't there yet, most importantly, from OS perspective. To elaborate on this: OS creates processes, assigns ids to them, assigns other physical resources to them s.a. association with namespaces (which later gives them user permissions, network access, virtual memory access, filesystem access etc.) and then these processes are associated with some GPU resource. If and when OS will start creating processes entirely on GPU, then it will make sense to talk about how GPUs are solving the problems of threads per core etc. For now it's a moot point. This is not said to discourage though. I really feel like CPU-centric model of what we call "computers" is not a good one going forward. The "periphery" is growing smarter with each generation, and wants to do its own computing, and spread its load somehow, and we keep coming up with ad hoc solutions that don't mix well with CPU-based concurrency, s.a. async I/O or CUDA. We really need a different concept of concurrency that would be more uniform and at the same time more flexible across different devices that can do work concurrently, and this, interface if you will, must come from the operating system, not as a user-space library to be truly effective.