4 ms·

Remember when they told us in CS class that it's better to design more efficient algorithms than to buy a faster CPU? Well here we are building nuclear reactor

Remember when they told us in CS class that it's better to design more efficient algorithms than to buy a faster CPU? Well here we are building nuclear reactors to run our brute force "scaled" LLMs. Really, really dumb.

DeepYogurt 1y ago
Big O? More like Big H2O (heavy water).... I'll see myself out.
JumpCrisscross 1y ago
> Remember when they told us in CS class that it's better to design more efficient algorithms than to buy a faster CPU? No? The tradeoff is entirely one between the value of labour versus the value of industry. If dev hours are cheap and CPUs expensive. If it’s the other way, which it is in AI, you buy more CPUs and GPUs.
- rhdhfjej 1y ago
  [flagged]
  - logicchains 1y ago
    And you would have been mocked by your peers for being so concieted that you'd dare to look down on other people for not inventing an algorithm that doesn't exist and for which there's no evidence it's even possible for one to exist.
    - rhdhfjej 1y ago
      [flagged]
      - oatsandsugar 1y ago
        Dig up
  - tootie 1y ago
    It's the difference between computer science and software engineering.
- estimator7292 1y ago
  This makes sense if and only if you entirely ignore all secondary and tertiary effects of your choices. Things like massively increased energy cost, strain on the grid, depriving local citizens of resources for your datacenter, and let's not forget ewaste, pollution from higher energy use, pollution caused by manufacturing more and more chips, pollution and cost of shipping more and more chips across the planet. Yeah, it's so cheap as to be nearly free.
  - codingrightnow 1y ago
    [flagged]
  - JumpCrisscross 1y ago
    > it's so cheap as to be nearly free Both chips and developer time are expensive. Massively so, both in direct cost and secondary and tertiary elements. (If you think hiring more developers to optimise code has no knock-on effects, I have a bridge to sell you.) There isn't an iron law about developer time being less valuable than chips. When chip progress stagnates, we tend towards optimising. When the developer pipeline is constrained, e.g. when a new frontier opens, we tend towards favouring exploration over optimisation. If a CS programme is teaching someone to always try to optimise an algorithm versus consider whether hardware might be the limitation, it's not a very good one. In this case, when it comes to AI, there is massive investment going into trying to find more efficient training and inference algorithms. Research which, ironically enough, generally requires access to energy.
  - yannyu 1y ago
    > Things like massively increased energy cost, strain on the grid This is a peculiarly USA-localized problem. For a large number of reasons, datacenters are going up all over the world now, and proportionally more of them are outside the US than has been the case historically. And a lot of these places have easier access to cheaper, cleaner power with modernized grids capable of handling it. > pollution from higher energy use Somewhat coincidentally as well, energy costs in China and the EU are projected to go down significantly over the the next 10 years due to solar and renewables, where it's not so clear that's going to happen in the US. As for the rest of the arguments around chip manufacturing and shipping and everything else, well, what do you expect? That we would just stop making chips? We only stopped using horses for transportation when we invented cars. I don't yet see what's going to replace our need for computing.
- utyop22 1y ago
  "Which it is in AI, you buy more CPUs and GPUs." Ermmm. what?
infecto 1y ago
Pretty exciting to me. Constraints breed innovation and it’s possible that the wave of AI leads to new breakthroughs on the green energy front. Edit: Amazing how anti-innovation and science folks are on HN.
logicchains 1y ago
There's no efficient algorithm for simulating a human brain, and you certainly haven't invented one so you've got absolutely no excuse to act smug about it. LLMs are already within an order of magnitude of the energy efficiency as the human brain, it's probably not possible to make them much more efficient algorithmically.
- irjustin 1y ago
  I'm really sad the core argument for the Matrix's existence doesn't hold up (it never did, just for me as a kid is all).
  - juliangamble 1y ago
    They started with a different, more brilliant idea, of using human brains as a giant neural net, then backed away from that: https://news.ycombinator.com/item?id=12508832 https://news.ycombinator.com/item?id=12508832
    - irjustin 1y ago
      oh wow that's cool. I do understand why they moved away from it. Battery is waaaayyyyy easier to understand for the layman and lay-kid (me).
- rhdhfjej 1y ago
  Your brain has a TDP of 15W while frontier LLMs require on the order of megawatts. That's 5-6 orders of magnitude difference, despite our semiconductors having a lithographic feature size that's also orders of magnitude smaller than our biological neurons. You should do some more research.
  - adrian_b 1y ago
    The TDP of a typical human brain is not 15 W, but 25 W, so about the same as for many notebook or mini-PC CPUs, but otherwise your argument stands. The idle power consumption of a human is around 100 W.
wmf 1y ago
There's a ton of research into more efficient AI algorithms. We've also seen that GPT-5 has better performance despite being no bigger than previous models. GPU/ASIC vendors are also increasing energy efficiency every generation. More datacenters will be needed despite these improvements because we're probably only using 1% of the potential of AI so far.
- zekrioca 1y ago
  Interesting, if GPUS/ASICS have been improving in energy efficiency, then why is it that total consumption has been exponentially increasing?
  - bobthepanda 1y ago
    Because so many people want to run the models. You see this in other sectors where demand outstrips improvements in economy. Individual planes use substantially less fuel than they did 50 years ago, because there are now fewer engines on planes and the remaining engines are also more efficient; but the growth in air travel has substantially outpaced that.
keepamovin 1y ago
You have a point. But it doesn't make sense to seek for the next unrealized breakthrough (low energy, brain-comparable power consumption) AI leap yet, when existing products are already so transformative. It will come, give it time.

DeepYogurt 1y ago

Big O? More like Big H2O (heavy water).... I'll see myself out.

JumpCrisscross 1y ago

> Remember when they told us in CS class that it's better to design more efficient algorithms than to buy a faster CPU? No? The tradeoff is entirely one between the value of labour versus the value of industry. If dev hours are cheap and CPUs expensive. If it’s the other way, which it is in AI, you buy more CPUs and GPUs.

rhdhfjej 1y ago
[flagged]
- logicchains 1y ago
  And you would have been mocked by your peers for being so concieted that you'd dare to look down on other people for not inventing an algorithm that doesn't exist and for which there's no evidence it's even possible for one to exist.
  - rhdhfjej 1y ago
    [flagged]
    - oatsandsugar 1y ago
      Dig up
- tootie 1y ago
  It's the difference between computer science and software engineering.
estimator7292 1y ago
This makes sense if and only if you entirely ignore all secondary and tertiary effects of your choices. Things like massively increased energy cost, strain on the grid, depriving local citizens of resources for your datacenter, and let's not forget ewaste, pollution from higher energy use, pollution caused by manufacturing more and more chips, pollution and cost of shipping more and more chips across the planet. Yeah, it's so cheap as to be nearly free.
- codingrightnow 1y ago
  [flagged]
- JumpCrisscross 1y ago
  > it's so cheap as to be nearly free Both chips and developer time are expensive. Massively so, both in direct cost and secondary and tertiary elements. (If you think hiring more developers to optimise code has no knock-on effects, I have a bridge to sell you.) There isn't an iron law about developer time being less valuable than chips. When chip progress stagnates, we tend towards optimising. When the developer pipeline is constrained, e.g. when a new frontier opens, we tend towards favouring exploration over optimisation. If a CS programme is teaching someone to always try to optimise an algorithm versus consider whether hardware might be the limitation, it's not a very good one. In this case, when it comes to AI, there is massive investment going into trying to find more efficient training and inference algorithms. Research which, ironically enough, generally requires access to energy.
- yannyu 1y ago
  > Things like massively increased energy cost, strain on the grid This is a peculiarly USA-localized problem. For a large number of reasons, datacenters are going up all over the world now, and proportionally more of them are outside the US than has been the case historically. And a lot of these places have easier access to cheaper, cleaner power with modernized grids capable of handling it. > pollution from higher energy use Somewhat coincidentally as well, energy costs in China and the EU are projected to go down significantly over the the next 10 years due to solar and renewables, where it's not so clear that's going to happen in the US. As for the rest of the arguments around chip manufacturing and shipping and everything else, well, what do you expect? That we would just stop making chips? We only stopped using horses for transportation when we invented cars. I don't yet see what's going to replace our need for computing.
utyop22 1y ago
"Which it is in AI, you buy more CPUs and GPUs." Ermmm. what?

infecto 1y ago

Pretty exciting to me. Constraints breed innovation and it’s possible that the wave of AI leads to new breakthroughs on the green energy front. Edit: Amazing how anti-innovation and science folks are on HN.

logicchains 1y ago

There's no efficient algorithm for simulating a human brain, and you certainly haven't invented one so you've got absolutely no excuse to act smug about it. LLMs are already within an order of magnitude of the energy efficiency as the human brain, it's probably not possible to make them much more efficient algorithmically.

irjustin 1y ago
I'm really sad the core argument for the Matrix's existence doesn't hold up (it never did, just for me as a kid is all).
- juliangamble 1y ago
  They started with a different, more brilliant idea, of using human brains as a giant neural net, then backed away from that: https://news.ycombinator.com/item?id=12508832 https://news.ycombinator.com/item?id=12508832
  - irjustin 1y ago
    oh wow that's cool. I do understand why they moved away from it. Battery is waaaayyyyy easier to understand for the layman and lay-kid (me).
rhdhfjej 1y ago
Your brain has a TDP of 15W while frontier LLMs require on the order of megawatts. That's 5-6 orders of magnitude difference, despite our semiconductors having a lithographic feature size that's also orders of magnitude smaller than our biological neurons. You should do some more research.
- adrian_b 1y ago
  The TDP of a typical human brain is not 15 W, but 25 W, so about the same as for many notebook or mini-PC CPUs, but otherwise your argument stands. The idle power consumption of a human is around 100 W.

wmf 1y ago

There's a ton of research into more efficient AI algorithms. We've also seen that GPT-5 has better performance despite being no bigger than previous models. GPU/ASIC vendors are also increasing energy efficiency every generation. More datacenters will be needed despite these improvements because we're probably only using 1% of the potential of AI so far.

zekrioca 1y ago
Interesting, if GPUS/ASICS have been improving in energy efficiency, then why is it that total consumption has been exponentially increasing?
- bobthepanda 1y ago
  Because so many people want to run the models. You see this in other sectors where demand outstrips improvements in economy. Individual planes use substantially less fuel than they did 50 years ago, because there are now fewer engines on planes and the remaining engines are also more efficient; but the growth in air travel has substantially outpaced that.

keepamovin 1y ago

You have a point. But it doesn't make sense to seek for the next unrealized breakthrough (low energy, brain-comparable power consumption) AI leap yet, when existing products are already so transformative. It will come, give it time.