5 ms·

Years ago I met a guy at a dinner party who had done some work for DARPA around super conducting computers. The big appeal was energy efficiency and speed: 100s

Years ago I met a guy at a dinner party who had done some work for DARPA around super conducting computers. The big appeal was energy efficiency and speed: 100s of GHz and ~99% energy reduction. One critical limitation is that information transfer from superconducting to regular wiring is relatively energy intensive, so a superconducting CPU won't actually result in advantages for input/output intensive processing. For processing heavy jobs though, if you don't have to shuffle data around the speed and energy savings will be very attractive.

Szpadel 2y ago
AFAIK current CPUs power usage for calculations is in pJ ( pico joule ) and that's why we are doing speculative execution because that's almost free the issue is that moving that data even to anything outside the core is cost about 100x that so the main every consumption for CPU is not in doing actual computation but in moving data around
- grishka 2y ago
  Isn't speculative execution there to do something potentially useful during clock cycles that would've been otherwise wasted?
  - dgacmu 2y ago
    It's mostly about avoiding the latency penalty. There's only so much ILP you can extract from the instruction stream before you get stuck waiting for a dependency. If you started executing that dependency speculatively, it will complete earlier, so you can launch the next instructions sooner. That lets you speed up single-threaded execution more by adding more functional units (since you can't really jam the clock speed any more).
    - vasco 2y ago
      This is exactly it. Speculative execution comes from instruction set design and basic constraints of branching. The energy consumption could make speculative execution prohibitive, but it's not "the" reason we do it.
      - credit_guy 2y ago
        Noob question here: is this the reason specialized chips can work so much better for AI applications? That the computations needed in a neural network are entirely deterministic and there is no need for branch prediction?
        vasco 2y ago
        Not really, it's more the massive parallelism. Branch prediction takes away something but mostly it's the parallelism. Each instruction you usually do in a GPU is a massive array in one go. In a cpu you need to use AVX type instructions and those are way more limited in the size of arrays they can process at once.
        credit_guy 2y ago
        Yes, the GPUs provide massive parallelism. An NVIDIA RTX 4090 has 16384 "cuda cores". Whatever these cuda cores are, they must be much, much smaller than a CPU core. They do computations though, and CPU cores do computations too. Why do the CPU cores need to be so much larger, so a CPU with more than 64 cores is rarely heard of, while GPUs have thousands of cores?
        vasco 2y ago
        Read about vector instructions a little bit and you'll see what I mean in the previous comment. A CPU has many many niche instructions it supports, it's way more flexible. A GPU is just trying to multiply the largest arrays possible as fast as possible, so the architecture becomes different. I don't think there's a quick way for you to grasp this without reading more about computer architecture and instruction sets, but you seem to be interested in it, so dive in :)
h0l0cube 2y ago
The tech described in the article features superconducting memory in combination with the 'SPU': > To complement the logic architecture, we also redesigned a compatible Josephson-junction-based SRAM. [...] > But there are also some striking differences. First, most of the chip is to be submerged in liquid helium for cooling to a mere 4 K. This includes the SPUs and SRAM, which depend on superconducting logic rather than CMOS, and are housed on an interposer board. Next, there is a glass bridge to a warmer area, a balmy 77 K that hosts the DRAM. The DRAM technology is not superconducting, but conventional silicon cooled down from room temperature, making it more efficient. From there, bespoke connectors lead data to and from the room-temperature world. ... with the idea of stacking this technology to put a 'supercomputer in a shoebox': > It is also straightforward to stack multiple boards of 3D superconducting chips on top of each other, leaving only a small space between them. We modeled a stack of 100 such boards, all operating within the same cooling environment and contained in a 20- by 20- by 12-centimeter volume, roughly the size of a shoebox. We calculated that this stack can perform 20 exaflops (in BF16 number format), 20 times the capacity of the largest supercomputer today. What’s more, the system promises to consume only 500 kilowatts of total power. This translates to energy efficiency one hundred times as high as the most efficient supercomputer today. ... or make datacenters a fraction of the size: > In addition, with this technology we can engineer data centers with much smaller footprints. Drastically smaller data centers can be placed close to their target applications, rather than being in some far-off football-stadium-size facility.

Szpadel 2y ago

AFAIK current CPUs power usage for calculations is in pJ ( pico joule ) and that's why we are doing speculative execution because that's almost free the issue is that moving that data even to anything outside the core is cost about 100x that so the main every consumption for CPU is not in doing actual computation but in moving data around

grishka 2y ago
Isn't speculative execution there to do something potentially useful during clock cycles that would've been otherwise wasted?
- dgacmu 2y ago
  It's mostly about avoiding the latency penalty. There's only so much ILP you can extract from the instruction stream before you get stuck waiting for a dependency. If you started executing that dependency speculatively, it will complete earlier, so you can launch the next instructions sooner. That lets you speed up single-threaded execution more by adding more functional units (since you can't really jam the clock speed any more).
  - vasco 2y ago
    This is exactly it. Speculative execution comes from instruction set design and basic constraints of branching. The energy consumption could make speculative execution prohibitive, but it's not "the" reason we do it.
    - credit_guy 2y ago
      Noob question here: is this the reason specialized chips can work so much better for AI applications? That the computations needed in a neural network are entirely deterministic and there is no need for branch prediction?
      - vasco 2y ago
        Not really, it's more the massive parallelism. Branch prediction takes away something but mostly it's the parallelism. Each instruction you usually do in a GPU is a massive array in one go. In a cpu you need to use AVX type instructions and those are way more limited in the size of arrays they can process at once.
        credit_guy 2y ago
        Yes, the GPUs provide massive parallelism. An NVIDIA RTX 4090 has 16384 "cuda cores". Whatever these cuda cores are, they must be much, much smaller than a CPU core. They do computations though, and CPU cores do computations too. Why do the CPU cores need to be so much larger, so a CPU with more than 64 cores is rarely heard of, while GPUs have thousands of cores?
        vasco 2y ago
        Read about vector instructions a little bit and you'll see what I mean in the previous comment. A CPU has many many niche instructions it supports, it's way more flexible. A GPU is just trying to multiply the largest arrays possible as fast as possible, so the architecture becomes different. I don't think there's a quick way for you to grasp this without reading more about computer architecture and instruction sets, but you seem to be interested in it, so dive in :)

h0l0cube 2y ago

The tech described in the article features superconducting memory in combination with the 'SPU': > To complement the logic architecture, we also redesigned a compatible Josephson-junction-based SRAM. [...] > But there are also some striking differences. First, most of the chip is to be submerged in liquid helium for cooling to a mere 4 K. This includes the SPUs and SRAM, which depend on superconducting logic rather than CMOS, and are housed on an interposer board. Next, there is a glass bridge to a warmer area, a balmy 77 K that hosts the DRAM. The DRAM technology is not superconducting, but conventional silicon cooled down from room temperature, making it more efficient. From there, bespoke connectors lead data to and from the room-temperature world. ... with the idea of stacking this technology to put a 'supercomputer in a shoebox': > It is also straightforward to stack multiple boards of 3D superconducting chips on top of each other, leaving only a small space between them. We modeled a stack of 100 such boards, all operating within the same cooling environment and contained in a 20- by 20- by 12-centimeter volume, roughly the size of a shoebox. We calculated that this stack can perform 20 exaflops (in BF16 number format), 20 times the capacity of the largest supercomputer today. What’s more, the system promises to consume only 500 kilowatts of total power. This translates to energy efficiency one hundred times as high as the most efficient supercomputer today. ... or make datacenters a fraction of the size: > In addition, with this technology we can engineer data centers with much smaller footprints. Drastically smaller data centers can be placed close to their target applications, rather than being in some far-off football-stadium-size facility.