4 ms·

Seems like a nice idea — instead of the stark CPU/GPU divide we have today, this would fit somewhere in the middle. Reminds me slightly of the Cell processor,

Seems like a nice idea — instead of the stark CPU/GPU divide we have today, this would fit somewhere in the middle. Reminds me slightly of the Cell processor, with its dedicated SPUs for fast processing, orchestrated by a traditional CPU. But we all saw how successful that was (: And that had some pretty big backing. Overcoming the inertia of the current computing hardware landscape is such a huge task. Maybe they can find some niche(s).

winwang 2y ago
I'd believe more in a heterogenous chip (e.g. MI300X, Apple M series, or even APUs) than in completely new chip tech.
- rbanffy 2y ago
  It’s still two separate ISAs.
  - winwang 2y ago
    I have a hard time believing that dealing with a singular ISA for two different compute cases would be an "obviously better" solution. I don't doubt that it's plausible/possible, especially if RISC-V + extensions are still simple, but then again, that's far outside my wheelhouse.
    - rbanffy 2y ago
      Even with a GPU sharing memory with your CPU, calling GPU code is never as simple as doing a jump to that code. You need to set up the code the GPU will run and then let it loose on the data.
mnky9800n 2y ago
Also thinking machines corporation https://en.wikipedia.org/wiki/Thinking_Machines_Corporation https://en.wikipedia.org/wiki/Thinking_Machines_Corporation
jbellis 2y ago
Isn't this where "NPUs" are going now?
aleph_minus_one 2y ago
> Reminds me slightly of the Cell processor, with its dedicated SPUs for fast processing, orchestrated by a traditional CPU. But we all saw how successful that was (: The success of the Cell is more subtle: - (It seems many) game developers hated it because it is so different to program than other CPUs of other game consoles of its time (in particular the CPU of the Xbox 360). For game studios, time to market and portability of the game to other consoles is important. - On the other hand, scientists who ported their high-performance numerical computations to the Cell seem to have loved it. Such software is often custom-built for the underlying hardware, and here cost (of the hardware) and possible speed are the measures on which to evaluate the hardware. Here the Cell processor of a PS3 cluster was much more competitive than other available solutions (GPGU did not really exist at this time).
- cedilla 2y ago
  Gabe Newell of Valve famously hated the Cell architecture, and I think that's very illustrative. He is of the generation of game devs that was very willing to try wild algorithms and hand-massage assembly and use all tricks to get 3D fast, so the PS3 should have been a perfect fit. But he did not like to have to start back at square one.
- amelius 2y ago
  Isn't that because game developers use conditional statements more, and scientists typically have a flow-graph that describes a computation and this computation doesn't have conditional parts? So it is a more natural fit?
  - aleph_minus_one 2y ago
    I don't know, in particular concerning the game developer perspective. But from my observation, scientists who develop high-performance computing algorithms often think much deeper about the mathematical structure of their problems than game developers do. I thus have a feeling that what you describe as "flow-graph that describes a computation" is rather a result of this deep analysis. I can easily imagine that this would partly also work for video games, but I would hypothesize either this is too much work that is not really rewarded in the game industry (the game industy is known ("crunch time") for having to churn out lots of new code fast), or if you are a lot into this kind of thinking, the game industry might not be the most rewarding place to work at.
CyberDildonics 2y ago
SIMD units already fit somewhere in the middle.
adrian_b 2y ago
What they say is far too vague, so it is impossible to know whether they have any new and original idea. It is well known that the CPU cores that are optimized for high single-threaded performance are bad for multithreaded tasks, because they have very poor performance per power and per area, so you cannot put many of them in a single package, because there are limits both for the die area and for the power dissipation. There are 3 solutions for this problem, all of which are used in many currently existing computers. 1. A hybrid CPU can be used, which has a few cores optimized for single-threaded performance and many cores optimized for multithreaded performance, like the Intel E-cores or the AMD compact cores. 2. One can have one or more accelerators for array operations, which are shared by the CPU cores and whose instruction streams are extracted from the instruction streams of the CPU cores (like in the CPUs from many decades ago the floating-point instructions were extracted from the CPU intruction streams and they were executed by floating-point coprocessors). The instructions executed by such accelerators must be defined in the ISA of the corresponding CPUs. Examples are the Arm SME/SME2 (Scalable Matrix Extension) and the Arm SSVE (Streaming Scalable Vector Extension) instruction sets. These ISA extensions are optional starting from Armv9.2-A or Armv8.7-A. AFAIK, for now only the recent Apple CPUs support them, but in the future the support for them might become widespread. 3. The last solution is to have an accelerator for array operations that has a mechanism independent from the CPU cores for fetching and decoding its own instruction stream. The CPU cores have to launch programs on such accelerators and get results when they are ready. Such completely independent accelerators are either parts of GPUs or they may be completely dedicated for computing tasks, when they no longer include the special-function graphics hardware. Any up-to-date laptop CPU already includes inside its package at least 2, if not all 3 of these solutions, to provide a good multithreaded performance. For servers, it is much less useful to have all these variants in a single package, because one can mix for instance one server with big cores with high single-threaded performance with many servers using much more compact cores per socket, for good multithreaded performance, and the servers can use multiple discrete GPUs per server. It is not clear with whom this "Flow Computing" wants to compete. They certainly cannot make better compact cores than Intel, AMD or Arm. They cannot make something like a SME accelerator, because that must be tightly integrated with the cores for which it functions as a coprocessor. So their "parallel processing units" can be only competitors for the existing GPUs or NPUs. Due to their origins in execution units for shader programs the current GPUs are not versatile enough. There still are programs that are easy to run on CPU cores but it is difficult to convert them to a form that can be executed by GPUs. So there would be a place for someone that could design an architecture more convenient than that of the current GPUs. However there is no indication in that article that there exists any problem for which the "Flow Computing" PPUs are better than the current GPUs or NPUs. If the PPUs have some kind of dataflow structure, then their application domain would be even more restricted than for the current GPUs and NPUs. EDIT: Now I have read their whitepaper "Design goals, advantages and benefits of Flow Computing", from HotChips. However, what that paper says about their patented architecture raises more questions than provides any answers. Their description of the PPUs is very similar to the description of Denelcor HEP from 1979. HEP (Heterogeneous Element Processor) was an experimental computer designed by Denelcor, Inc., which was intended to be a competitor for the supercomputers like Cray-1 (1976). While HEP was based on very good ideas, its practical implementation was very poor, using non-optimized and obsolete technology in comparison with Cray, so it has never demonstrated a competitive performance. The lead architect of HEP has later founded "Tera Computer Company", in 1987, which has designed computers based on the same ideas with HEP. Tera Computer had very modest results, but somehow it has succeeded in 2000 to buy the Cray Research division of Silicon Graphics, then it was renamed as Cray, Inc. (now a subsidiary of HPE). While Cray-1 and its predecessors (TI ASC and CDC STAR) were based on exploiting the parallelism of hardware pipelines with array operations, which can provide independent operations on distinct array elements, which can be executed in parallel in different pipeline stages, HEP was based on exploiting the parallelism of hardware pipelines with fine-grained multithreading, where independent instructions from distinct threads can be executed in parallel in different pipeline stages. HEP had multiple CPU cores ("core" was not a term used at that time). Each CPU core was a FGMT core, which could switch at each clock cycle between an extremely large number of threads. (FGMT is a term that has been introduced only much later, in 1996, with its abbreviation only in 1997; at the time of HEP, they used the term "fine-grained multiprogramming") The very large number of threads executed by each FGMT core (e.g. hundreds) can hide the latencies of data availability. The description of the "Flow Computing" PPUs is about the same as for HEP (1979), i.e. they appear to depend on FGMT with a very large number of threads (called "fibers" by Flow Computing) to hide the latencies. Unlike GPUs and NPUs, but like HEP, it seems the "Flow Computing" PPUs rely mainly on multithreading (a.k.a. TLP) to provide parallelism, and not on array operations (a.k.a. DLP). The revival of this old idea could actually be good, but the whitepaper does not provide any detail that would indicate whether they have found a better way to implement this.
RaftPeople 2y ago
> Reminds me slightly of the Cell processor I was thinking the same. Also the Tilera CPU with many cores and mesh network (back in mid 2000's, eventually bought by Nvidia and used in something, don't remember). Tangent: Back in early 2000's I had a hobby project (ALife with ANN brain) and I was looking for more computation. Multiple CPU's was not ideal, GPU wasn't ideal because the read/write/computation model only matched 1/2 of my ANN's flow and was a mismatch for the other half. I read about a new cpu and I ended up talking to one of the key guys from Tilera, I was pretty impressed they would take the time to talk to some random guy working on a hobby project. I asked about the performance of individual computational units (assuming custom could beat the industry) and he surprised me when he responded "nobody is going to beat Intel at integer, you won't get an increase from that perspective"

winwang 2y ago

I'd believe more in a heterogenous chip (e.g. MI300X, Apple M series, or even APUs) than in completely new chip tech.

rbanffy 2y ago
It’s still two separate ISAs.
- winwang 2y ago
  I have a hard time believing that dealing with a singular ISA for two different compute cases would be an "obviously better" solution. I don't doubt that it's plausible/possible, especially if RISC-V + extensions are still simple, but then again, that's far outside my wheelhouse.
  - rbanffy 2y ago
    Even with a GPU sharing memory with your CPU, calling GPU code is never as simple as doing a jump to that code. You need to set up the code the GPU will run and then let it loose on the data.

mnky9800n 2y ago

Also thinking machines corporation https://en.wikipedia.org/wiki/Thinking_Machines_Corporation https://en.wikipedia.org/wiki/Thinking_Machines_Corporation

jbellis 2y ago

Isn't this where "NPUs" are going now?

aleph_minus_one 2y ago

> Reminds me slightly of the Cell processor, with its dedicated SPUs for fast processing, orchestrated by a traditional CPU. But we all saw how successful that was (: The success of the Cell is more subtle: - (It seems many) game developers hated it because it is so different to program than other CPUs of other game consoles of its time (in particular the CPU of the Xbox 360). For game studios, time to market and portability of the game to other consoles is important. - On the other hand, scientists who ported their high-performance numerical computations to the Cell seem to have loved it. Such software is often custom-built for the underlying hardware, and here cost (of the hardware) and possible speed are the measures on which to evaluate the hardware. Here the Cell processor of a PS3 cluster was much more competitive than other available solutions (GPGU did not really exist at this time).

cedilla 2y ago
Gabe Newell of Valve famously hated the Cell architecture, and I think that's very illustrative. He is of the generation of game devs that was very willing to try wild algorithms and hand-massage assembly and use all tricks to get 3D fast, so the PS3 should have been a perfect fit. But he did not like to have to start back at square one.
amelius 2y ago
Isn't that because game developers use conditional statements more, and scientists typically have a flow-graph that describes a computation and this computation doesn't have conditional parts? So it is a more natural fit?
- aleph_minus_one 2y ago
  I don't know, in particular concerning the game developer perspective. But from my observation, scientists who develop high-performance computing algorithms often think much deeper about the mathematical structure of their problems than game developers do. I thus have a feeling that what you describe as "flow-graph that describes a computation" is rather a result of this deep analysis. I can easily imagine that this would partly also work for video games, but I would hypothesize either this is too much work that is not really rewarded in the game industry (the game industy is known ("crunch time") for having to churn out lots of new code fast), or if you are a lot into this kind of thinking, the game industry might not be the most rewarding place to work at.

CyberDildonics 2y ago

SIMD units already fit somewhere in the middle.

adrian_b 2y ago

What they say is far too vague, so it is impossible to know whether they have any new and original idea. It is well known that the CPU cores that are optimized for high single-threaded performance are bad for multithreaded tasks, because they have very poor performance per power and per area, so you cannot put many of them in a single package, because there are limits both for the die area and for the power dissipation. There are 3 solutions for this problem, all of which are used in many currently existing computers. 1. A hybrid CPU can be used, which has a few cores optimized for single-threaded performance and many cores optimized for multithreaded performance, like the Intel E-cores or the AMD compact cores. 2. One can have one or more accelerators for array operations, which are shared by the CPU cores and whose instruction streams are extracted from the instruction streams of the CPU cores (like in the CPUs from many decades ago the floating-point instructions were extracted from the CPU intruction streams and they were executed by floating-point coprocessors). The instructions executed by such accelerators must be defined in the ISA of the corresponding CPUs. Examples are the Arm SME/SME2 (Scalable Matrix Extension) and the Arm SSVE (Streaming Scalable Vector Extension) instruction sets. These ISA extensions are optional starting from Armv9.2-A or Armv8.7-A. AFAIK, for now only the recent Apple CPUs support them, but in the future the support for them might become widespread. 3. The last solution is to have an accelerator for array operations that has a mechanism independent from the CPU cores for fetching and decoding its own instruction stream. The CPU cores have to launch programs on such accelerators and get results when they are ready. Such completely independent accelerators are either parts of GPUs or they may be completely dedicated for computing tasks, when they no longer include the special-function graphics hardware. Any up-to-date laptop CPU already includes inside its package at least 2, if not all 3 of these solutions, to provide a good multithreaded performance. For servers, it is much less useful to have all these variants in a single package, because one can mix for instance one server with big cores with high single-threaded performance with many servers using much more compact cores per socket, for good multithreaded performance, and the servers can use multiple discrete GPUs per server. It is not clear with whom this "Flow Computing" wants to compete. They certainly cannot make better compact cores than Intel, AMD or Arm. They cannot make something like a SME accelerator, because that must be tightly integrated with the cores for which it functions as a coprocessor. So their "parallel processing units" can be only competitors for the existing GPUs or NPUs. Due to their origins in execution units for shader programs the current GPUs are not versatile enough. There still are programs that are easy to run on CPU cores but it is difficult to convert them to a form that can be executed by GPUs. So there would be a place for someone that could design an architecture more convenient than that of the current GPUs. However there is no indication in that article that there exists any problem for which the "Flow Computing" PPUs are better than the current GPUs or NPUs. If the PPUs have some kind of dataflow structure, then their application domain would be even more restricted than for the current GPUs and NPUs. EDIT: Now I have read their whitepaper "Design goals, advantages and benefits of Flow Computing", from HotChips. However, what that paper says about their patented architecture raises more questions than provides any answers. Their description of the PPUs is very similar to the description of Denelcor HEP from 1979. HEP (Heterogeneous Element Processor) was an experimental computer designed by Denelcor, Inc., which was intended to be a competitor for the supercomputers like Cray-1 (1976). While HEP was based on very good ideas, its practical implementation was very poor, using non-optimized and obsolete technology in comparison with Cray, so it has never demonstrated a competitive performance. The lead architect of HEP has later founded "Tera Computer Company", in 1987, which has designed computers based on the same ideas with HEP. Tera Computer had very modest results, but somehow it has succeeded in 2000 to buy the Cray Research division of Silicon Graphics, then it was renamed as Cray, Inc. (now a subsidiary of HPE). While Cray-1 and its predecessors (TI ASC and CDC STAR) were based on exploiting the parallelism of hardware pipelines with array operations, which can provide independent operations on distinct array elements, which can be executed in parallel in different pipeline stages, HEP was based on exploiting the parallelism of hardware pipelines with fine-grained multithreading, where independent instructions from distinct threads can be executed in parallel in different pipeline stages. HEP had multiple CPU cores ("core" was not a term used at that time). Each CPU core was a FGMT core, which could switch at each clock cycle between an extremely large number of threads. (FGMT is a term that has been introduced only much later, in 1996, with its abbreviation only in 1997; at the time of HEP, they used the term "fine-grained multiprogramming") The very large number of threads executed by each FGMT core (e.g. hundreds) can hide the latencies of data availability. The description of the "Flow Computing" PPUs is about the same as for HEP (1979), i.e. they appear to depend on FGMT with a very large number of threads (called "fibers" by Flow Computing) to hide the latencies. Unlike GPUs and NPUs, but like HEP, it seems the "Flow Computing" PPUs rely mainly on multithreading (a.k.a. TLP) to provide parallelism, and not on array operations (a.k.a. DLP). The revival of this old idea could actually be good, but the whitepaper does not provide any detail that would indicate whether they have found a better way to implement this.

RaftPeople 2y ago

> Reminds me slightly of the Cell processor I was thinking the same. Also the Tilera CPU with many cores and mesh network (back in mid 2000's, eventually bought by Nvidia and used in something, don't remember). Tangent: Back in early 2000's I had a hobby project (ALife with ANN brain) and I was looking for more computation. Multiple CPU's was not ideal, GPU wasn't ideal because the read/write/computation model only matched 1/2 of my ANN's flow and was a mismatch for the other half. I read about a new cpu and I ended up talking to one of the key guys from Tilera, I was pretty impressed they would take the time to talk to some random guy working on a hobby project. I asked about the performance of individual computational units (assuming custom could beat the industry) and he surprised me when he responded "nobody is going to beat Intel at integer, you won't get an increase from that perspective"