6 ms·

This is great! I've waited for chips approaching something like 256 to 1024 cores since the 90s. These CPUs still leave a lot to be desired, but I sense that th

This is great! I've waited for chips approaching something like 256 to 1024 cores since the 90s. These CPUs still leave a lot to be desired, but I sense that there's been a shift towards better uses of transistor count than just single-threaded performance. These are still roughly 10 times more expensive than they should be because of their memory architecture. I'd vote to drop the idea of busses to external memories and switch to local memories, then have the cores self-organize by using web metaphors like content-addressable memory (CAM) to handle caching. Basically get rid of all of the cache coherence hardware and treat each core-memory as its own computer. The hardware that wasn't scalable could go to hardware-accelerated hashing for the CAM. And a somewhat controversial opinion - I'd probably drop 64 bit also and either emulate 64 bit math on an 8/16/32 bit processor, or switch to arbitrary precision. That's because the number of cores is scalable and quickly dwarfs bits calculated per cycle. So we'd take say a 10% performance hit for a 100% increase in the number of cores, something like that. This would probably need to be tested in simulation to know where the threshold is, maybe 64 cores or something. Similar arguments could be used for clock speed and bus width, etc.

rbanffy 5y ago
> I'd probably drop 64 bit also There are a lot of other nice things that would have to go along that. A 32-bit linear address space is not enough for a lot of the things we do today, specially not in servers. Having some memory dedicated to a given core is clever, however, provided we have the required affinity settings to match (moving a task to a different core would imply copying the scratchpad to the new core and would be extremely costly - much more than the cache misses we account for in current kernels) What I would drop immediately is ISA compatibility. I have no use for it provided I can compile my code on the new ISA.
- zackmorris 5y ago
  Oh I just meant that the CPU might have a 64 bit interface and address space, but be 32 bit or less internally. Mostly the microcode would provide the emulation, but maybe a few instructions like 64 bit multiply would have dedicated hardware. If that turns out to be the bulk of the hardware needed in the first place, then maybe my idea wouldn't work. Maybe a hybrid design, half the transistors for a 5% performance loss or something. Ya I agree, ISA is kind of an antiquated concept. Intel chips basically emulate x86 instructions in microcode today, as a glaring example.
  - SV_BubbleTime 5y ago
    Or… just use 64bit through out? I’m not following what you are trying to save by adding a mixed system and emulation. I can’t imagine dropping 64bit to pump cores up.
    - zackmorris 5y ago
      Sorry, I'm coming at this from a pretty low level. Things like ALUs sometimes use lookup tables to do things like N bit multiply, instead of carrying bits serially like in a 1 bit handheld calculator. But there are fundamental limits to that where maybe there just isn't transistor budget to multiply more than 8 or 16 bits or something. Or there are propagation delay issues where the circuit just can't get any faster, so instructions must be pipelined to approach something like 1 instruction per cycle. So it really depends on the architecture. Sometimes it makes sense to have "native" 64 bit math, and sometimes it makes sense to compose a 64 bit operation from many 8/16/32 bit operations. In this case, I would opt for using as few transistors as possible per-core in order to fit more cores on the chip. Then expose a 64 bit architecture, but implement that as 8/16/32 bit software loops in the microcode. So maybe a multiply might take 2 or 4 or 8 cycles or something instead of 1. In practice superscalar architecture, pipelining, caching, even hyper-threading, can hide much of that because processors today usually run dozens or even hundreds of times faster than main memory. So I highly doubt that chips today actually implement every single 64 bit instruction in hardware. Granted, my knowledge around this is over 20 years old, but it comes from studying stuff like MIPS processors, so is about as fundamental as it gets. And single-threaded performance has only roughly tripled since the 90s. My guess is that they use more lookup tables now since transistors are basically free. At the cost of never going back and checking how effectively they're using their transistor budget for actual computation, which results in the CPUs we have today that run many thousands of times slower than they could. We see this in how GPUs absolutely dwarf the performance of CPUs.
  - rbanffy 5y ago
    I have the impression reorder buffers took a huge amount of space, specially in irregularly sized ISAs like x86. Simplifying the ISA could free up a lot of space for 64-bit multiply units.
dmitrygr 5y ago
> treat each core-memory as its own computer. There was a company about a decade back that did this. I seem to remember it was useful for web serving. Bought by AMD. Not sure what happened next. Look them up. Name was SeaMicro
- rbanffy 5y ago
  They were discontinued. I ported some enablement software from Python 2 to 3 for it as part of Canonical's MaaS platform (we had one in the OpenStack Integration Lab, IIRC). I wouldn't mind finding one on my driveway ;-)
dragontamer 5y ago
> These are still roughly 10 times more expensive than they should be because of their memory architecture. I'd vote to drop the idea of busses to external memories and switch to local memories, then have the cores self-organize by using web metaphors like content-addressable memory (CAM) to handle caching. Basically get rid of all of the cache coherence hardware and treat each core-memory as its own computer. The hardware that wasn't scalable could go to hardware-accelerated hashing for the CAM. If you want "many cores" and "get rid of cache-coherence hardware", its called a GPU. Yes, a lot of those "cores" are SIMD-lanes, at least by NVidia / AMD naming conventions. But GPU SIMD-lanes have memory-fetching hardware that operates per-lane, so you approximate the effects of a many-many core computer. ------- Japanese companies are experimenting with more CPUs though. PEZY "villages" are all proper CPUs IIRC, but this architecture isn't very popular outside of Japan. In terms of the global market, your best bet is in fact a GPU. The Fujitsu ARM supercomputer was also ARM-based + HBM2. But once again, that's a specific Japanese supercomputer and not very popular outside of Japan. It is available though.
- freemint 5y ago
  Japanese Companies such as NEC are also selling Vector Computers such as the NEC SX-Aurora TSUBASA Plugin cards.
  - dragontamer 5y ago
    Yeah. I like to read the whitepapers from these Japanese companies but I have no practical experience with them. The TSUBASA cards look like they're somewhere in between CPU and GPU. But its hard for me to understand it without actually playing with them directly.
    - freemint 5y ago
      The German weather service uses them extensively so they are probably really decent with PDEs.
samus 5y ago
Number of cores is just another metric to optimize for. What counts in the end is whether it can efficiently and quickly deal with the load it is expected to handle. Many cores are great for mostly independent tasks, but performance will suffer as soon as communication is required. Making chip architectures more distributed seems to be the state of the art at the moment, but this doesn't mean we will suddenly be able to escape Amdahl's Law. To be specific, for inherently serial applications where we are absolutely interested in getting the result ASAP, single-thread performance remains crucial.

rbanffy 5y ago

> I'd probably drop 64 bit also There are a lot of other nice things that would have to go along that. A 32-bit linear address space is not enough for a lot of the things we do today, specially not in servers. Having some memory dedicated to a given core is clever, however, provided we have the required affinity settings to match (moving a task to a different core would imply copying the scratchpad to the new core and would be extremely costly - much more than the cache misses we account for in current kernels) What I would drop immediately is ISA compatibility. I have no use for it provided I can compile my code on the new ISA.

zackmorris 5y ago
Oh I just meant that the CPU might have a 64 bit interface and address space, but be 32 bit or less internally. Mostly the microcode would provide the emulation, but maybe a few instructions like 64 bit multiply would have dedicated hardware. If that turns out to be the bulk of the hardware needed in the first place, then maybe my idea wouldn't work. Maybe a hybrid design, half the transistors for a 5% performance loss or something. Ya I agree, ISA is kind of an antiquated concept. Intel chips basically emulate x86 instructions in microcode today, as a glaring example.
- SV_BubbleTime 5y ago
  Or… just use 64bit through out? I’m not following what you are trying to save by adding a mixed system and emulation. I can’t imagine dropping 64bit to pump cores up.
  - zackmorris 5y ago
    Sorry, I'm coming at this from a pretty low level. Things like ALUs sometimes use lookup tables to do things like N bit multiply, instead of carrying bits serially like in a 1 bit handheld calculator. But there are fundamental limits to that where maybe there just isn't transistor budget to multiply more than 8 or 16 bits or something. Or there are propagation delay issues where the circuit just can't get any faster, so instructions must be pipelined to approach something like 1 instruction per cycle. So it really depends on the architecture. Sometimes it makes sense to have "native" 64 bit math, and sometimes it makes sense to compose a 64 bit operation from many 8/16/32 bit operations. In this case, I would opt for using as few transistors as possible per-core in order to fit more cores on the chip. Then expose a 64 bit architecture, but implement that as 8/16/32 bit software loops in the microcode. So maybe a multiply might take 2 or 4 or 8 cycles or something instead of 1. In practice superscalar architecture, pipelining, caching, even hyper-threading, can hide much of that because processors today usually run dozens or even hundreds of times faster than main memory. So I highly doubt that chips today actually implement every single 64 bit instruction in hardware. Granted, my knowledge around this is over 20 years old, but it comes from studying stuff like MIPS processors, so is about as fundamental as it gets. And single-threaded performance has only roughly tripled since the 90s. My guess is that they use more lookup tables now since transistors are basically free. At the cost of never going back and checking how effectively they're using their transistor budget for actual computation, which results in the CPUs we have today that run many thousands of times slower than they could. We see this in how GPUs absolutely dwarf the performance of CPUs.
- rbanffy 5y ago
  I have the impression reorder buffers took a huge amount of space, specially in irregularly sized ISAs like x86. Simplifying the ISA could free up a lot of space for 64-bit multiply units.

dmitrygr 5y ago

> treat each core-memory as its own computer. There was a company about a decade back that did this. I seem to remember it was useful for web serving. Bought by AMD. Not sure what happened next. Look them up. Name was SeaMicro

rbanffy 5y ago
They were discontinued. I ported some enablement software from Python 2 to 3 for it as part of Canonical's MaaS platform (we had one in the OpenStack Integration Lab, IIRC). I wouldn't mind finding one on my driveway ;-)

dragontamer 5y ago

> These are still roughly 10 times more expensive than they should be because of their memory architecture. I'd vote to drop the idea of busses to external memories and switch to local memories, then have the cores self-organize by using web metaphors like content-addressable memory (CAM) to handle caching. Basically get rid of all of the cache coherence hardware and treat each core-memory as its own computer. The hardware that wasn't scalable could go to hardware-accelerated hashing for the CAM. If you want "many cores" and "get rid of cache-coherence hardware", its called a GPU. Yes, a lot of those "cores" are SIMD-lanes, at least by NVidia / AMD naming conventions. But GPU SIMD-lanes have memory-fetching hardware that operates per-lane, so you approximate the effects of a many-many core computer. ------- Japanese companies are experimenting with more CPUs though. PEZY "villages" are all proper CPUs IIRC, but this architecture isn't very popular outside of Japan. In terms of the global market, your best bet is in fact a GPU. The Fujitsu ARM supercomputer was also ARM-based + HBM2. But once again, that's a specific Japanese supercomputer and not very popular outside of Japan. It is available though.

freemint 5y ago
Japanese Companies such as NEC are also selling Vector Computers such as the NEC SX-Aurora TSUBASA Plugin cards.
- dragontamer 5y ago
  Yeah. I like to read the whitepapers from these Japanese companies but I have no practical experience with them. The TSUBASA cards look like they're somewhere in between CPU and GPU. But its hard for me to understand it without actually playing with them directly.
  - freemint 5y ago
    The German weather service uses them extensively so they are probably really decent with PDEs.

samus 5y ago

Number of cores is just another metric to optimize for. What counts in the end is whether it can efficiently and quickly deal with the load it is expected to handle. Many cores are great for mostly independent tasks, but performance will suffer as soon as communication is required. Making chip architectures more distributed seems to be the state of the art at the moment, but this doesn't mean we will suddenly be able to escape Amdahl's Law. To be specific, for inherently serial applications where we are absolutely interested in getting the result ASAP, single-thread performance remains crucial.