4 ms·

Nothing forces the rebound effect to dominate. Computers grow cheaper, we rebound by buying ones with higher capacity, but the overall price still shrinks. I be

by Arnt 1y ago

Nothing forces the rebound effect to dominate. Computers grow cheaper, we rebound by buying ones with higher capacity, but the overall price still shrinks. I bet the computer you used to post today cost much less than Colossus. Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead. You can stream films via 5G that you might not have done via 4G, but you might've streamed via WLAN or perhaps cat5 cable instead. The rebound effect doesn't force 5G to use more power than WLAN/GBE. Or more power than driving to a cinema, if you want to compare really widely. The film you stream makes it comparable, not?

bilekas 1y ago
> Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead Am I missing something or has the need to vast GPU horsepower been solved ? Those requirements were not in DC's before and they're only going up. Whatever way you look at it, there's got to be an increase in power consumption somewhere no ?
- Arnt 1y ago
  Not necessarily, no. You can pick and choose your comparisons, and make an incease appear or not. Take weather forecasts as an example. Weather forecasting uses massively powerful computers today. If you compare that forecasting with the lack of forecasts two hundred years ago there obviously is an increase in power usage (no electricity was used then) or there obviously isn't (today's result is something we didn't have then, so it would be an apples-to-nothing comparison). If you say "the GPUs are using power now that they weren't using before" you're implicitly doing the former kind of comparison. Which is obviously correct or obviously wrong ;)
- timschmidt 1y ago
  GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt. AI is still very near the beginning of the optimization process. We're still using (relatively) general purpose processors to run it. Dedicated accelerators are beginning to appear. Many software optimizations will be found. FPGAs and ASICs will be designed and fabbed. Process nodes will continue to shrink. Moore will continue to exponentially decrease costs over time as with all other workloads.
  - philipwhiuk 1y ago
    > Moore will continue to exponentially decrease costs over time as with all other workloads. There's absolutely no guarantee of this. The continuation of Moore's law is far from certain (NVIDIA think it's dead already).
    - timschmidt 1y ago
      > NVIDIA think it's dead already Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too. Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice. Until then, they can sit with the group of people who've been vocally wrong about moore's law's end for the last 50 years. Our chips are still overwhelmingly 2D in design, just a few dozen layers thick but billions of transistors wide. We have quite a ways to go based on a first principles analysis alone. And indeed, that's what chip engineers like Jim Keller say: https://www.youtube.com/watch?v=c01BlUDIlK4 https://www.youtube.com/watch?v=c01BlUDIlK4 So ask yourself how it benefits Jensen to convince you otherwise.
      - adgjlsfhk1 1y ago
        progress continues, but at far slower rates than they used to. nvidia has gained ~6x density in the past 9 years (1080 to 5090), while a doubling every 2 years would be >20x density in 9 years. the past 6 years (3090) are even worse with only a 3x of density
        timschmidt 1y ago
        Moore's law says nothing about density. "The complexity for minimum component costs has increased at a rate of roughly a factor of two per year." Density is one way in which industry has met this observation over decades. New processes (NMOS, CMOS, etc) is another. New packaging techniques (flip chip, BGA, etc). New substrates. There's no limit to process innovation. Nvidia's also optimizing their designs for things other than minimum component cost. I.e. higher clock speeds, lower temperatures, lower power consumption, etc. It may seem like I'm picking a nit here, but such compromises are fundamental to the cost efficiency Moore was referencing. All data I've seen, once fully considered, indicates that Moore's law is healthy and thriving.
Analemma_ 1y ago
There is some limit to the rebound effect because people only have so many hours in the day, but we’re nowhere near the ceiling of how much AI compute people could use. Note how many people pay for the $200/month plans from Anthropic, OAI etc. and still hit limits because they constantly spend $8000 worth of tokens letting the agents burn and churn. It’s pretty obvious that as compute gets cheaper via hardware improvements and power buildout, usage is going to climb exponentially as people go “eh, let the agent just run on autopilot, who cares if it takes 2MM tokens to do [simple task]”. I think for the foreseeable future we should consider the rebound effect in this sector to be in full force and not expect any decreases in power usage for a long time.
everdrive 1y ago
>Nothing forces the rebound effect to dominate. Human nature does. We're like a gas, and we fill to expand the space we're in. If technology uses less power, in general, we'll just use more of it until we hit whatever natural limits are present. (usually cost, or availability) I'm not sure I'm a proponent of usage taxes, but they definitely have the right idea; people will just keep doing more things until it becomes too expensive or they are otherwise restricted. The problem you run into is how the public reacts when "they" are trying to force a bunch of limitations on you that you didn't previously need to live with. It's politically impossible, even in a case where it's the right choice.
- Arnt 1y ago
  I don't understand why "we're like a gas, and expand to fill the space we're in". What makes the simile apply to e.g. AI or 5G when it doesn't apply to others, e.g. computer prices?
  - everdrive 1y ago
    I think the Apple ARM chips are a good example. They're fantastically more efficient and fantastically powerful. We _could_ take this incredible platform and say "we can probably do personal computing on 3-5 watts if we really focus on efficiency." But we don't do that. With more powerful chips, websites and apps and operating systems will get less efficient, bigger, more bloated. If there's any slack in the system we'll just take it up with crap. The chips will be faster next year so why bother making things more efficient? Repeat this process forever, and we eat up all of our efficiency gains.
    - Arnt 1y ago
      My Apple ARM laptop has 24+ hours battery lifetime in practice, three times as much as the best laptop I had in the 2000-2015 period, and it's lighter than most of those laptops too. (Can't remember the battery lifetime of my 2015-2019 laptop.) Clearly not all efficiency gains have been eaten up.
      - 1y ago
palata 1y ago
> Nothing forces the rebound effect to dominate. Not sure what to say to that. Yeah, it would be great if we didn't put so much resources into destroying our own world. I agree. The fact is that rebound effect very much dominates everything we do. I'm not saying it should, I'm saying it does. It's an observation.
- Arnt 1y ago
  I think you're telling me that graphs like these always point upwards, right? https://ourworldindata.org/grapher/per-capita-energy-use?tab=line&time=earliest..2024 https://ourworldindata.org/grapher/per-capita-energy-use?tab... My own impression is that sometimes the aggregate total grows and sometimes it doesn't. And when it grows, sometimes that's because the rebound effect dominates.
  - palata 1y ago
    Read the part in your link that says "Here, energy refers to primary energy". Primary energy in the US ignores the primary energy used in China for goods that end up being imported in the US.
    - Arnt 1y ago
      I googled briefly now, and found https://tradingeconomics.com/united-states/imports/china https://tradingeconomics.com/united-states/imports/china. Looking at that (note that the X axis is not zero) it's clear that Chinese imports cannot account for any significant part of the changes to the the USA graph on https://ourworldindata.org/grapher/per-capita-energy-use?tab=line&time=earliest..2024 https://ourworldindata.org/grapher/per-capita-energy-use?tab...

by Arnt 1y ago

bilekas 1y ago

> Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead Am I missing something or has the need to vast GPU horsepower been solved ? Those requirements were not in DC's before and they're only going up. Whatever way you look at it, there's got to be an increase in power consumption somewhere no ?

Arnt 1y ago
Not necessarily, no. You can pick and choose your comparisons, and make an incease appear or not. Take weather forecasts as an example. Weather forecasting uses massively powerful computers today. If you compare that forecasting with the lack of forecasts two hundred years ago there obviously is an increase in power usage (no electricity was used then) or there obviously isn't (today's result is something we didn't have then, so it would be an apples-to-nothing comparison). If you say "the GPUs are using power now that they weren't using before" you're implicitly doing the former kind of comparison. Which is obviously correct or obviously wrong ;)
timschmidt 1y ago
GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt. AI is still very near the beginning of the optimization process. We're still using (relatively) general purpose processors to run it. Dedicated accelerators are beginning to appear. Many software optimizations will be found. FPGAs and ASICs will be designed and fabbed. Process nodes will continue to shrink. Moore will continue to exponentially decrease costs over time as with all other workloads.
- philipwhiuk 1y ago
  > Moore will continue to exponentially decrease costs over time as with all other workloads. There's absolutely no guarantee of this. The continuation of Moore's law is far from certain (NVIDIA think it's dead already).
  - timschmidt 1y ago
    > NVIDIA think it's dead already Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too. Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice. Until then, they can sit with the group of people who've been vocally wrong about moore's law's end for the last 50 years. Our chips are still overwhelmingly 2D in design, just a few dozen layers thick but billions of transistors wide. We have quite a ways to go based on a first principles analysis alone. And indeed, that's what chip engineers like Jim Keller say: https://www.youtube.com/watch?v=c01BlUDIlK4 https://www.youtube.com/watch?v=c01BlUDIlK4 So ask yourself how it benefits Jensen to convince you otherwise.
    - adgjlsfhk1 1y ago
      progress continues, but at far slower rates than they used to. nvidia has gained ~6x density in the past 9 years (1080 to 5090), while a doubling every 2 years would be >20x density in 9 years. the past 6 years (3090) are even worse with only a 3x of density
      - timschmidt 1y ago
        Moore's law says nothing about density. "The complexity for minimum component costs has increased at a rate of roughly a factor of two per year." Density is one way in which industry has met this observation over decades. New processes (NMOS, CMOS, etc) is another. New packaging techniques (flip chip, BGA, etc). New substrates. There's no limit to process innovation. Nvidia's also optimizing their designs for things other than minimum component cost. I.e. higher clock speeds, lower temperatures, lower power consumption, etc. It may seem like I'm picking a nit here, but such compromises are fundamental to the cost efficiency Moore was referencing. All data I've seen, once fully considered, indicates that Moore's law is healthy and thriving.

Analemma_ 1y ago

There is some limit to the rebound effect because people only have so many hours in the day, but we’re nowhere near the ceiling of how much AI compute people could use. Note how many people pay for the $200/month plans from Anthropic, OAI etc. and still hit limits because they constantly spend $8000 worth of tokens letting the agents burn and churn. It’s pretty obvious that as compute gets cheaper via hardware improvements and power buildout, usage is going to climb exponentially as people go “eh, let the agent just run on autopilot, who cares if it takes 2MM tokens to do [simple task]”. I think for the foreseeable future we should consider the rebound effect in this sector to be in full force and not expect any decreases in power usage for a long time.

everdrive 1y ago

>Nothing forces the rebound effect to dominate. Human nature does. We're like a gas, and we fill to expand the space we're in. If technology uses less power, in general, we'll just use more of it until we hit whatever natural limits are present. (usually cost, or availability) I'm not sure I'm a proponent of usage taxes, but they definitely have the right idea; people will just keep doing more things until it becomes too expensive or they are otherwise restricted. The problem you run into is how the public reacts when "they" are trying to force a bunch of limitations on you that you didn't previously need to live with. It's politically impossible, even in a case where it's the right choice.

Arnt 1y ago
I don't understand why "we're like a gas, and expand to fill the space we're in". What makes the simile apply to e.g. AI or 5G when it doesn't apply to others, e.g. computer prices?
- everdrive 1y ago
  I think the Apple ARM chips are a good example. They're fantastically more efficient and fantastically powerful. We _could_ take this incredible platform and say "we can probably do personal computing on 3-5 watts if we really focus on efficiency." But we don't do that. With more powerful chips, websites and apps and operating systems will get less efficient, bigger, more bloated. If there's any slack in the system we'll just take it up with crap. The chips will be faster next year so why bother making things more efficient? Repeat this process forever, and we eat up all of our efficiency gains.
  - Arnt 1y ago
    My Apple ARM laptop has 24+ hours battery lifetime in practice, three times as much as the best laptop I had in the 2000-2015 period, and it's lighter than most of those laptops too. (Can't remember the battery lifetime of my 2015-2019 laptop.) Clearly not all efficiency gains have been eaten up.
    - 1y ago

palata 1y ago

> Nothing forces the rebound effect to dominate. Not sure what to say to that. Yeah, it would be great if we didn't put so much resources into destroying our own world. I agree. The fact is that rebound effect very much dominates everything we do. I'm not saying it should, I'm saying it does. It's an observation.

Arnt 1y ago
I think you're telling me that graphs like these always point upwards, right? https://ourworldindata.org/grapher/per-capita-energy-use?tab=line&time=earliest..2024 https://ourworldindata.org/grapher/per-capita-energy-use?tab... My own impression is that sometimes the aggregate total grows and sometimes it doesn't. And when it grows, sometimes that's because the rebound effect dominates.
- palata 1y ago
  Read the part in your link that says "Here, energy refers to primary energy". Primary energy in the US ignores the primary energy used in China for goods that end up being imported in the US.
  - Arnt 1y ago
    I googled briefly now, and found https://tradingeconomics.com/united-states/imports/china https://tradingeconomics.com/united-states/imports/china. Looking at that (note that the X axis is not zero) it's clear that Chinese imports cannot account for any significant part of the changes to the the USA graph on https://ourworldindata.org/grapher/per-capita-energy-use?tab=line&time=earliest..2024 https://ourworldindata.org/grapher/per-capita-energy-use?tab...