4 ms·

I think people like author are positive about us, humanity, being able to build AI or something being very close to that. I am not. From the energy efficiency

I think people like author are positive about us, humanity, being able to build AI or something being very close to that. I am not. From the energy efficiency perspective human brain is very, very effective computational machine. Computers are not. Thinking about scale of infrastructure of network of computers being able to achieve similar capabilities and its energy consumption... it would be enormous. With big infrastructure comes high need of maintenance. This is costly and requires a lot of people just to prevent it from breaking down. With a lot of people being in one place, there socioeconomical cost, production, transportation needs to be build around such center. If you have centralized system, you are prone to attack from adversaries. In short I do not think we even close to what author is afraid of. We just closer to beginning to understand what is the need to actually start to think about building AI - if ever possible at all.

Chris2048 1y ago
> he energy efficiency perspective human brain is very, very effective computational machine Can you explain why you think that? Very often, mechanical efficiency outperforms biological. Humans have existed for thougsands of years, neurons even longer. Computers and AI and relatively recent, we haven't really begun to explore optimisation possibilities.
- rcxdude 1y ago
  It may be possible to optimise silicon further, but the brain does all of its work with less than a hundred watts, while the silicon closest to its capabilities needs more like tens of kW.
  - deleted 1y ago
    [deleted]
  - Ukv 1y ago
    > while the silicon closest to its capabilities needs more like tens of kW. I think looking at power consumption for the very edge of what technology is just barely capble of may be misleading, since that's inherently at one extreme of the current cost-capability trade-off curve[0] and stands to drop the most drastically from efficiency improvements. You can now run models equivalent in capability to initial version of ChatGPT on sub-20w chips, for instance. Or, looking over a longer timeframe, we can now do far more on a 1-milliwatt chip[1] than on the 150kW ENIAC[2]. [0]: https://i.imgur.com/GydBGRG.png https://i.imgur.com/GydBGRG.png [1]: https://spectrum.ieee.org/syntiant-chip-plays-doom https://spectrum.ieee.org/syntiant-chip-plays-doom [2]: https://cse.engin.umich.edu/about/history/eniac-display/ https://cse.engin.umich.edu/about/history/eniac-display/
    - rcxdude 1y ago
      Well, sure, but the capabilities of the edge of tech is still not matching the human brain. And the fact the you can run older models on less power also doesn't say anything for certain. I'm not saying it won't happen, I'm saying it's not happened and it's not certain it will.
      - deleted 1y ago
        [deleted]
- daveguy 1y ago
  The human brain runs on about 20 watts of power -- the entire body on 80 watts (at rest). Those numbers are at least good within an order of magnitude. The largest supercomputer consumes 29 megawatts of power for 1.7 exaflops. And the largest supercomputers are nowhere near the flexible generality of a human brain -- they're calculating FFTs for the test. The amount of parallelism in the human brain is enormous. Not just each neuron, but each synapse has computational capacity. That means ~10^14 computational units or 100 trillion processing units -- on about 20 watts. That doesn't even touch the bandwidth issues. Getting the sensory input in and out of the brain plus the bandwidth to get all of the processing signals between each neuron is at least another petabit per second. So, on bandwidth capacity alone we are 25+ years away (assuming the last 25 years of growth continues). And in humans that comes with 18 years of training at that massive bandwidth and computational power. Also, we have no idea what a general intelligence algorithm looks like. We are just now getting multimodal LLMs. From the computational/bandwidth perspective we are still 30 years from a computer being able to process The information a single human brain does, except while consuming 29+ megawatts of energy. If you had to feed a human 29 megawatts worth of power no business would be profitable. Humans wouldn't even survive. Sorry, but the notion that we are close to AGI because we have good word predictors is fantasy. But, there will be some amazing natural language human-computer interface improvements over the next 10 years!
  - lioeters 1y ago
    The next logical step, perhaps ethically questionable, seems to be growing human brains for computational purposes (parallel or quantum) with high bandwidth and very efficient power consumption.
    - daveguy 1y ago
      Unfortunately we are much farther from growing a human brain than we are from a scaling up an LLM to a 29 megawatt-consuming behemoth. With growing a brain, we barely know where to begin. Not in terms of growing a few neurons in a petri dish. Nourishing the complex interconnecting structure of neurons that is a human brain is nowhere even on the horizon. Much less growing the structure from cells. At least with the LLM/AI techniques we have control over the entire processing pipeline. And I agree, that is an ethical minefield.
- southernplaces7 1y ago
  >we haven't really begun to explore optimisation possibilities. So you're questioning the above comment's argument based on a hand-wavy claim about completely speculative future possibilities? As it stands, there's no disagreeing with the human brain's energy efficiency for all the computing it does in so many ways that AI can't even begin to match. This to not even speak of the whole unknown territory of whatever it is that gives us consciousness.
  - Chris2048 1y ago
    Is it speculative to suggest that technology will improve? No. > whatever it is that gives us consciousness talk about hand-wavy; "consciousness" might not be a real thing. You might as well ask if AI has a soul.
    - southernplaces7 1y ago
      It is indeed speculative to not just suggest that technology will improve but make specific claims about how and in what way based on no clear connection to any current development. Also, there's nothing hand wavy about pointing out -aside from all the vastly efficient parallelism and generalist computing that the brain does with absurdly minimal power needs- that it also seems to be where our consciousness is housed. You can go ahead and naval gaze about "how do we know if we're conscious? How do we know an LLM isn't?" but I certainly feel conscious, and so do you, and we both demonstrably have self-directed agency that indicates this widely, solidly accepted phenomenon, and is very distinct from anything any LLM can demonstrably do, regardless of what AIbros like to claim about consciousness not being real. Arguments like these remind me of relativist fall-back idiocies of asking "but what is a spoon" whenever confronted with any hard counterargument to their completely speculative claims about X or Y.
awanderingmind 1y ago
I hope you are right - the point about energy efficiency is certainly spot on, and I do think it is possible that people are getting carried away by analogies when discussing the topic (I wrote something about that too, but will avoid linking to it here to avoid excessive self-promotion). That said, the article doesn't assume such a thing will happen soon, just that it may happen at some time in the future. That could be centuries away - I would still argue the end result is something to be concerned about.

Chris2048 1y ago

> he energy efficiency perspective human brain is very, very effective computational machine Can you explain why you think that? Very often, mechanical efficiency outperforms biological. Humans have existed for thougsands of years, neurons even longer. Computers and AI and relatively recent, we haven't really begun to explore optimisation possibilities.

rcxdude 1y ago
It may be possible to optimise silicon further, but the brain does all of its work with less than a hundred watts, while the silicon closest to its capabilities needs more like tens of kW.
- deleted 1y ago
  [deleted]
- Ukv 1y ago
  > while the silicon closest to its capabilities needs more like tens of kW. I think looking at power consumption for the very edge of what technology is just barely capble of may be misleading, since that's inherently at one extreme of the current cost-capability trade-off curve[0] and stands to drop the most drastically from efficiency improvements. You can now run models equivalent in capability to initial version of ChatGPT on sub-20w chips, for instance. Or, looking over a longer timeframe, we can now do far more on a 1-milliwatt chip[1] than on the 150kW ENIAC[2]. [0]: https://i.imgur.com/GydBGRG.png https://i.imgur.com/GydBGRG.png [1]: https://spectrum.ieee.org/syntiant-chip-plays-doom https://spectrum.ieee.org/syntiant-chip-plays-doom [2]: https://cse.engin.umich.edu/about/history/eniac-display/ https://cse.engin.umich.edu/about/history/eniac-display/
  - rcxdude 1y ago
    Well, sure, but the capabilities of the edge of tech is still not matching the human brain. And the fact the you can run older models on less power also doesn't say anything for certain. I'm not saying it won't happen, I'm saying it's not happened and it's not certain it will.
    - deleted 1y ago
      [deleted]
daveguy 1y ago
The human brain runs on about 20 watts of power -- the entire body on 80 watts (at rest). Those numbers are at least good within an order of magnitude. The largest supercomputer consumes 29 megawatts of power for 1.7 exaflops. And the largest supercomputers are nowhere near the flexible generality of a human brain -- they're calculating FFTs for the test. The amount of parallelism in the human brain is enormous. Not just each neuron, but each synapse has computational capacity. That means ~10^14 computational units or 100 trillion processing units -- on about 20 watts. That doesn't even touch the bandwidth issues. Getting the sensory input in and out of the brain plus the bandwidth to get all of the processing signals between each neuron is at least another petabit per second. So, on bandwidth capacity alone we are 25+ years away (assuming the last 25 years of growth continues). And in humans that comes with 18 years of training at that massive bandwidth and computational power. Also, we have no idea what a general intelligence algorithm looks like. We are just now getting multimodal LLMs. From the computational/bandwidth perspective we are still 30 years from a computer being able to process The information a single human brain does, except while consuming 29+ megawatts of energy. If you had to feed a human 29 megawatts worth of power no business would be profitable. Humans wouldn't even survive. Sorry, but the notion that we are close to AGI because we have good word predictors is fantasy. But, there will be some amazing natural language human-computer interface improvements over the next 10 years!
- lioeters 1y ago
  The next logical step, perhaps ethically questionable, seems to be growing human brains for computational purposes (parallel or quantum) with high bandwidth and very efficient power consumption.
  - daveguy 1y ago
    Unfortunately we are much farther from growing a human brain than we are from a scaling up an LLM to a 29 megawatt-consuming behemoth. With growing a brain, we barely know where to begin. Not in terms of growing a few neurons in a petri dish. Nourishing the complex interconnecting structure of neurons that is a human brain is nowhere even on the horizon. Much less growing the structure from cells. At least with the LLM/AI techniques we have control over the entire processing pipeline. And I agree, that is an ethical minefield.
southernplaces7 1y ago
>we haven't really begun to explore optimisation possibilities. So you're questioning the above comment's argument based on a hand-wavy claim about completely speculative future possibilities? As it stands, there's no disagreeing with the human brain's energy efficiency for all the computing it does in so many ways that AI can't even begin to match. This to not even speak of the whole unknown territory of whatever it is that gives us consciousness.
- Chris2048 1y ago
  Is it speculative to suggest that technology will improve? No. > whatever it is that gives us consciousness talk about hand-wavy; "consciousness" might not be a real thing. You might as well ask if AI has a soul.
  - southernplaces7 1y ago
    It is indeed speculative to not just suggest that technology will improve but make specific claims about how and in what way based on no clear connection to any current development. Also, there's nothing hand wavy about pointing out -aside from all the vastly efficient parallelism and generalist computing that the brain does with absurdly minimal power needs- that it also seems to be where our consciousness is housed. You can go ahead and naval gaze about "how do we know if we're conscious? How do we know an LLM isn't?" but I certainly feel conscious, and so do you, and we both demonstrably have self-directed agency that indicates this widely, solidly accepted phenomenon, and is very distinct from anything any LLM can demonstrably do, regardless of what AIbros like to claim about consciousness not being real. Arguments like these remind me of relativist fall-back idiocies of asking "but what is a spoon" whenever confronted with any hard counterargument to their completely speculative claims about X or Y.

awanderingmind 1y ago

I hope you are right - the point about energy efficiency is certainly spot on, and I do think it is possible that people are getting carried away by analogies when discussing the topic (I wrote something about that too, but will avoid linking to it here to avoid excessive self-promotion). That said, the article doesn't assume such a thing will happen soon, just that it may happen at some time in the future. That could be centuries away - I would still argue the end result is something to be concerned about.