4 ms·

The OS I work on currently supports a maximum of 254 logical cpus. If we ran on the 9754 and had to ignore two entire threads, we'd be giving up a maximum of 0.

by 0x0203 3y ago

The OS I work on currently supports a maximum of 254 logical cpus. If we ran on the 9754 and had to ignore two entire threads, we'd be giving up a maximum of 0.78% performance. I remember being excited about the first dual core systems not that long ago... Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that. There seemed to be a fundamental shift between the Pentium 4 era and multi-core. Then between 8 core and many core. Big/Little also working its way in there. What's the next major innovation that's going to drive this ever-increasing performance limit? Or is there a theoretical maximum a single chip can do?

pixl97 3y ago
> Makes me wonder how long this trend of just packing more cores into the chip can continue, Latest GPUs have 10,000+ CUDA Cores, so, if you can paralyze the work, evidently pretty damned far. Of course we'll need ever faster versions of PCI-E to come out to feed these beasts with enough data they don't sit idle most of the time.
- calaphos 3y ago
  The cuda cores of Nvidia GPUs are closer to fp32 units in vector ALUs than CPU cores capable of operating independently in parallel. Following that definition a modern CPU core would have dozens of "cuda cores" as well (although far fewer than GPUs optimized for that kind of workload) More comparable would be the ~130 streaming Multiprocessors of a H100.
  - imtringued 3y ago
    The fact that each cuda core also has its own instruction pointer is quite misleading. You would think that this lets it run different instructions per cuda core but the opposite is the case. The driver uses these instruction pointers for finer scheduling granularity. That is cool but is not the same. https://stackoverflow.com/questions/58071834/why-does-each-thread-have-its-own-instruction-address-counter-inside-a-warp https://stackoverflow.com/questions/58071834/why-does-each-t...
    - adrian_b 3y ago
      In the NVIDIA peculiar language, which seems to have been chosen with the only purpose of causing confusion, because all the names invented by NVIDIA are applied to things that already had traditional names for many decades, warp means thread and thread means SIMD lane. NVIDIA has never given a good explanation about what they mean by the "instruction pointer" that belongs to each NVIDIA "thread". It certainly does not mean what in means normally, i.e. a special register that contains the address from where the next instruction will be fetched for execution. I believe that this "instruction pointer" refers to a register where the actual instruction pointer is saved when a "thread" is stalled because it has diverged into two branches after a condition test and only one of the branches continues to be executed, while the other branch must be executed later, with the complementary predicate. These saved instruction pointers are presumably used for scheduling the "threads" to be executed by the SIMD lanes provided by the hardware, in such a way as to satisfy the cross-lane dependencies.
hulitu 3y ago
> Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that It really depends on the cache and memory architecture. You need to be able to feed data to the CPU efficiently.
jiehong 3y ago
And close to 50% on a double socket with 512 logical cpus ;)
rbanffy 3y ago
> Or is there a theoretical maximum a single chip can do? There's a point where you won't be able to pack more transistors on a chip, but if Cerebras' work is an indication, chips themselves can get quite large. But remember these chips are not designed with desktop users in mind. These chips are more similar to the ones inside very large computers that are built to host hundreds of simultaneous virtual machines on behalf of hundreds of users. Sadly, we won't see desktops based on these unless we custom order them.
- thulle 3y ago
  Linking a quick intro to these 850000 core beasts: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-... And STH recently posted an article about how they're used: https://www.servethehome.com/100m-usd-cerebras-ai-cluster-makes-it-the-post-legacy-silicon-ai-winner/ https://www.servethehome.com/100m-usd-cerebras-ai-cluster-ma...

by 0x0203 3y ago

pixl97 3y ago

> Makes me wonder how long this trend of just packing more cores into the chip can continue, Latest GPUs have 10,000+ CUDA Cores, so, if you can paralyze the work, evidently pretty damned far. Of course we'll need ever faster versions of PCI-E to come out to feed these beasts with enough data they don't sit idle most of the time.

calaphos 3y ago
The cuda cores of Nvidia GPUs are closer to fp32 units in vector ALUs than CPU cores capable of operating independently in parallel. Following that definition a modern CPU core would have dozens of "cuda cores" as well (although far fewer than GPUs optimized for that kind of workload) More comparable would be the ~130 streaming Multiprocessors of a H100.
- imtringued 3y ago
  The fact that each cuda core also has its own instruction pointer is quite misleading. You would think that this lets it run different instructions per cuda core but the opposite is the case. The driver uses these instruction pointers for finer scheduling granularity. That is cool but is not the same. https://stackoverflow.com/questions/58071834/why-does-each-thread-have-its-own-instruction-address-counter-inside-a-warp https://stackoverflow.com/questions/58071834/why-does-each-t...
  - adrian_b 3y ago
    In the NVIDIA peculiar language, which seems to have been chosen with the only purpose of causing confusion, because all the names invented by NVIDIA are applied to things that already had traditional names for many decades, warp means thread and thread means SIMD lane. NVIDIA has never given a good explanation about what they mean by the "instruction pointer" that belongs to each NVIDIA "thread". It certainly does not mean what in means normally, i.e. a special register that contains the address from where the next instruction will be fetched for execution. I believe that this "instruction pointer" refers to a register where the actual instruction pointer is saved when a "thread" is stalled because it has diverged into two branches after a condition test and only one of the branches continues to be executed, while the other branch must be executed later, with the complementary predicate. These saved instruction pointers are presumably used for scheduling the "threads" to be executed by the SIMD lanes provided by the hardware, in such a way as to satisfy the cross-lane dependencies.

hulitu 3y ago

> Makes me wonder how long this trend of just packing more cores into the chip can continue, and where it will go after that It really depends on the cache and memory architecture. You need to be able to feed data to the CPU efficiently.

jiehong 3y ago

And close to 50% on a double socket with 512 logical cpus ;)

rbanffy 3y ago

> Or is there a theoretical maximum a single chip can do? There's a point where you won't be able to pack more transistors on a chip, but if Cerebras' work is an indication, chips themselves can get quite large. But remember these chips are not designed with desktop users in mind. These chips are more similar to the ones inside very large computers that are built to host hundreds of simultaneous virtual machines on behalf of hundreds of users. Sadly, we won't see desktops based on these unless we custom order them.

thulle 3y ago
Linking a quick intro to these 850000 core beasts: https://www.anandtech.com/show/16626/cerebras-unveils-wafer-scale-engine-two-wse2-26-trillion-transistors-100-yield https://www.anandtech.com/show/16626/cerebras-unveils-wafer-... And STH recently posted an article about how they're used: https://www.servethehome.com/100m-usd-cerebras-ai-cluster-makes-it-the-post-legacy-silicon-ai-winner/ https://www.servethehome.com/100m-usd-cerebras-ai-cluster-ma...