6 ms·

We've known about this for a long time. Everyone expected it to happen. There are some key upcoming technologies that have the potential to cause a step in scal

by kayson 3y ago

We've known about this for a long time. Everyone expected it to happen. There are some key upcoming technologies that have the potential to cause a step in scaling (CFETs, backside power delivery) but it's still not going to be anywhere near Moore's law levels. I think this is part of why GPU power is skyrocketing and why Apple, Qualcomm, and the like are trying to shift towards services.

Reason077 3y ago
It helps that those services are a great recurring revenue stream, too.
moffkalast 3y ago
Well the main problem is resistance isn't it? Most of the power "used" is to get electrons to flow fast enough for the logic gates to settle for a specific clock frequency and the resistive losses to heat. The only real way forward that isn't a temporary workaround seems finding a new type of semiconductor that has lower overall resistance than silicon. Whoever figures out how to dope graphene and produce wafers without defects will probably make trillions.
- timerol 3y ago
  tl;dr New materials can help, but "resistive losses" aren't really the driving factor. The energy is a mix of leakage current and active current. Leakage current can be thought of as resistance - it's how much current flows through a transistor that's off. This can be better based on the material, but gets harder with smaller transistors. (Thinking about quantum tunneling as a resistance is good to get intuition, but not good enough to help solve the problem. A material with a lower bulk resistivity will not help here.) Active current is based on capacitance. Each FET has a little capacitor that needs to be charged and discharged every time the logic is switched - that adds up. Lowering the capacitance of each FET would reduce the energy required to switch it, but generally comes with bad tradeoffs. High-k dielectrics increase the capacitance, all other things being equal. But all other things are not equal, and they are used to create better performing FETs with lower power leakage.
  - saltcured 3y ago
    I thought leakage current would be the "DC" loss that is independent of frequency, like we had in old bipolar logic. Isn't it fair to characterize the cmos/fet switching losses as resistance to moving the charges around? I understand leakage will go up if we increase voltages to support higher switching speeds, but aren't there still a lot of losses that happen with logic transitions and reduce when the states are stable, even if voltages are held constant? I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced? I.e. much of the waste is resistive losses while charging and discharging those gates.
    - timerol 3y ago
      > Isn't it fair to characterize the cmos/fet switching losses as resistance to moving the charges around? Not really. It makes more sense to think about it as filling and emptying capacitors. You are charging the gate capacitance up to the supply voltage, then dumping that charge to discharge the gate to 0 again. The energy of each capacitance that gets charged and dumped is CV^2/2, which happens for each logic transition. > I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced? If there was no resistance when distributing charge, it would help a bit, but not enough to change the clock frequency by more than 20%, assuming that the fantasy superconducting-fet had normal leakage and gate capacitance.
      - saltcured 3y ago
        So the charge is work and the discharge is waste? I guess I am entertaining the idea of an idealized Maxwell-demon CMOS circuit, if we could bounce the charge between gates with very little work to just pump the charge back and forth.
        timerol 3y ago
        That's a reasonable way to think about it - you take energy from the supply voltage to charge the gate capacitor when the logic line goes high, then dump it when the logic line goes low. If you had a lossless bidirectional voltage converter circuit for each gate capacitance, then you could charge the capacitor from the supply and discharge it back into the supply, removing any switching losses.
- LordDragonfang 3y ago
  When talking about resistance and materials, it's also important to note that silicon has relatively low optimal operating temperatures compared to some of the other semiconductors available. This limits the amount of voltage you can pump into it (because the resistance mean higher V leads to heat), and voltage correlates with clock frequency. GaN has already seen success in chargers, and silicon carbide is another promising material. We can't achieve the low level of defects needed for small process nodes yet, though. Disclaimer: I'm not a material scientist, so this is probably only partly correct.
Yoric 3y ago
Well, and photonics, quantum computing, etc. But they're not there yet.
qayxc 3y ago
> I think this is part of why GPU power is skyrocketing and why Apple, Qualcomm, and the like are trying to shift towards services. IMHO it's only a very small part of why GPU power consumption is going up. The main reason is the completely unnecessary chase for the performance crown. From personal testing: my GPU manages to get 95% of its peak performance while being power limited to 80%. So the in order to squeeze the last 5% of performance out of the device, 20% more power is pushed through it. It stays above 99% peak performance while being power limited to ~87%. But even just looking at the raw numbers paints a different picture. About 12 years ago, a high-end GPU (e.g. GTX 480) had a power draw of 250W at a theoretical peak FP32 performance of 1,345 GFLOPS. This year's RTX 4070 has a theoretical peak performance of 29.15 TFOPS at 200W, so we went from 5.38 GFLOPS/W to 145.75 GFLOPS/W in 12 years - a 27x improvement in efficiency and a ~22x improvement in raw performance. Now let's compare that to the numbers from a decade ago: a GTX 580 from 2010 had a power rating of 244W at 49.41 GTexel/s. A Geforce2 Ultra from 2000 used about 10W at 2.0 GTexel/s. So we went from 0.2 GTexel/s/W to - you've guessed it - 0.2 GTexel/s/W, so same efficiency with a ~27x increase in performance over a decade, though the efficiency is only a guess, since neither GFLOPS nor official power draw figures are readily available for 2000-era hardware. Fast forward a few years so we can get reliable power draw numbers and comparable performance in GFLOPS, we have the high end GeForce 8800 GTX at 155W for 345.6 GFLOPS in 2006. Ten years later, the comparable model would have been the GTX 1080 from 2016 with 180W at 8.873 TFLOPS. So 2.2 GFLOPS/W versus 49.3 GFLOPS/W or a 22x increase in efficiency and a ~26x increase in performance over the course of a decade. So during the past 23 years, power efficiency steadily improved, while raw performance increase also showed no signs of slow down in the GPU space. This is given the same generous time frames, to account for the occasional generational leap.
- pixl97 3y ago
  I would also think that GPU workloads have something to do with it. Almost none are serial workflows, and instead highly parallel work. GPU workloads will eventually run into the same scaling limits. That is we will be unable to speed up each execution unit any further, or the primary work we give the GPU will not be able to be split into more threads and accomplish useful work.
deleted 3y ago
[deleted]
Kon-Peki 3y ago
> We've known about this for a long time. Everyone expected it to happen. Absolutely. The 2006 "A View from Berkeley" is still a great paper [1]. And we still have a long ways to go on this recommendation: > To maximize application efficiency, programming models should support a wide range of data types and successful models of parallelism We are still stuck in the winner-take-all mindset when it comes to software development. [1] https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...
Animats 3y ago
> GPU power is skyrocketing Not quite. NVidia's entry level price/performance has not improved much since 2016. What's skyrocketing is the price of the top of the line models.
- NovaDudely 3y ago
  When the Voodoo 2 card launched it was $299. Adjusted for inflation that would be about $550. The latest 4090RTX is RRP of $1599. High end is now becoming a case of throwing us much money at the problem as possible.

by kayson 3y ago

Reason077 3y ago

It helps that those services are a great recurring revenue stream, too.

moffkalast 3y ago

Well the main problem is resistance isn't it? Most of the power "used" is to get electrons to flow fast enough for the logic gates to settle for a specific clock frequency and the resistive losses to heat. The only real way forward that isn't a temporary workaround seems finding a new type of semiconductor that has lower overall resistance than silicon. Whoever figures out how to dope graphene and produce wafers without defects will probably make trillions.

timerol 3y ago
tl;dr New materials can help, but "resistive losses" aren't really the driving factor. The energy is a mix of leakage current and active current. Leakage current can be thought of as resistance - it's how much current flows through a transistor that's off. This can be better based on the material, but gets harder with smaller transistors. (Thinking about quantum tunneling as a resistance is good to get intuition, but not good enough to help solve the problem. A material with a lower bulk resistivity will not help here.) Active current is based on capacitance. Each FET has a little capacitor that needs to be charged and discharged every time the logic is switched - that adds up. Lowering the capacitance of each FET would reduce the energy required to switch it, but generally comes with bad tradeoffs. High-k dielectrics increase the capacitance, all other things being equal. But all other things are not equal, and they are used to create better performing FETs with lower power leakage.
- saltcured 3y ago
  I thought leakage current would be the "DC" loss that is independent of frequency, like we had in old bipolar logic. Isn't it fair to characterize the cmos/fet switching losses as resistance to moving the charges around? I understand leakage will go up if we increase voltages to support higher switching speeds, but aren't there still a lot of losses that happen with logic transitions and reduce when the states are stable, even if voltages are held constant? I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced? I.e. much of the waste is resistive losses while charging and discharging those gates.
  - timerol 3y ago
    > Isn't it fair to characterize the cmos/fet switching losses as resistance to moving the charges around? Not really. It makes more sense to think about it as filling and emptying capacitors. You are charging the gate capacitance up to the supply voltage, then dumping that charge to discharge the gate to 0 again. The energy of each capacitance that gets charged and dumped is CV^2/2, which happens for each logic transition. > I realize it we can't move charges around for free, but in some fantasy superconducting-fet logic circuit, wouldn't the power consumption be reduced? If there was no resistance when distributing charge, it would help a bit, but not enough to change the clock frequency by more than 20%, assuming that the fantasy superconducting-fet had normal leakage and gate capacitance.
    - saltcured 3y ago
      So the charge is work and the discharge is waste? I guess I am entertaining the idea of an idealized Maxwell-demon CMOS circuit, if we could bounce the charge between gates with very little work to just pump the charge back and forth.
      - timerol 3y ago
        That's a reasonable way to think about it - you take energy from the supply voltage to charge the gate capacitor when the logic line goes high, then dump it when the logic line goes low. If you had a lossless bidirectional voltage converter circuit for each gate capacitance, then you could charge the capacitor from the supply and discharge it back into the supply, removing any switching losses.
LordDragonfang 3y ago
When talking about resistance and materials, it's also important to note that silicon has relatively low optimal operating temperatures compared to some of the other semiconductors available. This limits the amount of voltage you can pump into it (because the resistance mean higher V leads to heat), and voltage correlates with clock frequency. GaN has already seen success in chargers, and silicon carbide is another promising material. We can't achieve the low level of defects needed for small process nodes yet, though. Disclaimer: I'm not a material scientist, so this is probably only partly correct.

Yoric 3y ago

Well, and photonics, quantum computing, etc. But they're not there yet.

qayxc 3y ago

> I think this is part of why GPU power is skyrocketing and why Apple, Qualcomm, and the like are trying to shift towards services. IMHO it's only a very small part of why GPU power consumption is going up. The main reason is the completely unnecessary chase for the performance crown. From personal testing: my GPU manages to get 95% of its peak performance while being power limited to 80%. So the in order to squeeze the last 5% of performance out of the device, 20% more power is pushed through it. It stays above 99% peak performance while being power limited to ~87%. But even just looking at the raw numbers paints a different picture. About 12 years ago, a high-end GPU (e.g. GTX 480) had a power draw of 250W at a theoretical peak FP32 performance of 1,345 GFLOPS. This year's RTX 4070 has a theoretical peak performance of 29.15 TFOPS at 200W, so we went from 5.38 GFLOPS/W to 145.75 GFLOPS/W in 12 years - a 27x improvement in efficiency and a ~22x improvement in raw performance. Now let's compare that to the numbers from a decade ago: a GTX 580 from 2010 had a power rating of 244W at 49.41 GTexel/s. A Geforce2 Ultra from 2000 used about 10W at 2.0 GTexel/s. So we went from 0.2 GTexel/s/W to - you've guessed it - 0.2 GTexel/s/W, so same efficiency with a ~27x increase in performance over a decade, though the efficiency is only a guess, since neither GFLOPS nor official power draw figures are readily available for 2000-era hardware. Fast forward a few years so we can get reliable power draw numbers and comparable performance in GFLOPS, we have the high end GeForce 8800 GTX at 155W for 345.6 GFLOPS in 2006. Ten years later, the comparable model would have been the GTX 1080 from 2016 with 180W at 8.873 TFLOPS. So 2.2 GFLOPS/W versus 49.3 GFLOPS/W or a 22x increase in efficiency and a ~26x increase in performance over the course of a decade. So during the past 23 years, power efficiency steadily improved, while raw performance increase also showed no signs of slow down in the GPU space. This is given the same generous time frames, to account for the occasional generational leap.

pixl97 3y ago
I would also think that GPU workloads have something to do with it. Almost none are serial workflows, and instead highly parallel work. GPU workloads will eventually run into the same scaling limits. That is we will be unable to speed up each execution unit any further, or the primary work we give the GPU will not be able to be split into more threads and accomplish useful work.

deleted 3y ago

[deleted]

Kon-Peki 3y ago

> We've known about this for a long time. Everyone expected it to happen. Absolutely. The 2006 "A View from Berkeley" is still a great paper [1]. And we still have a long ways to go on this recommendation: > To maximize application efficiency, programming models should support a wide range of data types and successful models of parallelism We are still stuck in the winner-take-all mindset when it comes to software development. [1] https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-183.html https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...

Animats 3y ago

> GPU power is skyrocketing Not quite. NVidia's entry level price/performance has not improved much since 2016. What's skyrocketing is the price of the top of the line models.

NovaDudely 3y ago
When the Voodoo 2 card launched it was $299. Adjusted for inflation that would be about $550. The latest 4090RTX is RRP of $1599. High end is now becoming a case of throwing us much money at the problem as possible.