4 ms·

I've said this before, at the root of all these technological promises lies a perpetual motion machine. They're all selling the reversal of thermodynamics. Any

I've said this before, at the root of all these technological promises lies a perpetual motion machine. They're all selling the reversal of thermodynamics. Any system complex enough to be useful has to be embedded in an ever more complex system. The age of mobile phone internet rests on the shoulders of an immense and enormously complex supply chain. LLMs are capturing low entropy from data online and distilling it for you while producing a shitton of entropy on the backend. All the water and energy dissipated at data centers, all the supply chains involved in building GPUs at the rate we are building. There will be no magical moment when it's gonna yield more low entropy than what we put in on the other side as training data, electricity and clean water. When companies sell ideas like 'AGI' or 'self driving cars' they are essentially promising you can do away with the complexity surrounding a complex solution. They are promising they can deliver low entropy on a tap without paying for it in increased entropy elsewhere. It's physically impossible. You want human intelligence to do work, you need to deal with all the complexities of psychology, economics and politics. You want complex machines to do autonomous work, you need an army of people behind it. What AGI promises is, you can replace the army of people with another more complex machine. It's a big bald faced lie. You can't do away with the complexity. Someone will have to handle it.

ben_w 2y ago
> It's physically impossible Your brain is proof to the contrary. AGI means different things to everyone, but a human brain definitely counts as "general intelligence", that implemented in silicon is enough to get basically all the things promised by AGI: if that's done at the 20 watts per brain that biology manages, then all of humanity can be simulated within the power envelope of the USA electrical grid… three times over.
- namaria 2y ago
  > a human brain definitely counts as "general intelligence", that implemented in silicon is enough to get basically all the things promised by AGI: if that's done at the 20 watts per brain that biology manages, then all of humanity can be simulated within the power envelope of the USA electrical grid… three times over. So far the only thing that has been proven is we can get low entropy from all the low entropy we've published on the internet. Will it get to a point where models can give us more low entropy than what is present in the training data? Categorically: no.
  - ben_w 2y ago
    You are using "entropy" in a way I do not recognise. Whatever you mean, our brains prove it's possible to have a system that uses 20 watts to demonstrate human-level intelligence.
    - namaria 2y ago
      Whatever you mean, it took 13 billion years and the whole universe as far as we can tell to create the 20 watt human intelligence brain. And it doesn't take 20 watt of energy to maintain humans, it takes a whole ecosystem capable of sustaining a human population. And for us to operate at our current level of information processing it takes a whole planetary civilization. So no you haven't proved anything by saying that a human brain consumes only 20 watts of energy. We spread an awful lot of entropy around that has to be kept away from our delicate biological and social systems. You're positing a way to create human intelligence-like in a bottle, that's the same as speculating about the shape of a reality where we have FTL travel or teleportation or whatever else you fancy. If we're talking about what current ML/AI can do, they can extract patterns from training data and than apply those patterns to other inputs. This can give us great automation, but it won't give us anything better than the training data, solve physics, global warming, poverty or give us human intelligence in a chip. Whatever quantity Q of entropy in the training data, the total output will be more Q all accounted for. That's true for humans and machines. No shape of possible AGI will give us any output with less Q than the combination of inputs had. The dream that a machine will solve all problems that humanity can't hinges on negating that, which goes against thermodynamics. As it stands, the planet cannot cope with all the entropy we're spreading around. It will eventually collapse civilization/the ecosystem, whatever buckles first, from the excess entropy. Because global warming, poverty, or ignorance is just entropy. Disorder. Things not being just so as we need them to be.
      - ben_w 2y ago
        > Whatever you mean, it took 13 billion years and the whole universe as far as we can tell to create the 20 watt human intelligence brain. It's grossly unreasonable to include the entire history of the universe given the Earth only formed about 4 billion years ago; and given that evolution wasn't even aiming for intelligence, even starting from our common ancestors being small rodents 65 million years ago is wildly overstating the effort required — even the evolution of primates is too far back without intentional selective breeding. > You're positing a way to create human intelligence-like in a bottle, that's the same as speculating about the shape of a reality where we have FTL travel or teleportation or whatever else you fancy. FTL may well well be impossible. If you seriously think human intelligence is impossible, then you also think you don't exist: You, yourself, are a human like intelligence in a bottle. The bottle being your skull. > This can give us great automation, but it won't solve physics, global warming, poverty or give us human intelligence in a chip. AI has already been in widesprad use in physics for a while now, well before the current zeitgeist of LLMs. There's a Y Combinator startup: "Charge Robotics is building robots that automate the most labor-intensive parts of solar construction". Poverty has many causes, some of which are already being reduced or resolved by existing systems — and that's been the case since one of the ancestor companies of IBM, Tabulating Machine Company, was doing punched cards for the US census. As for human intelligence on a chip? Well, (1) it's been quite a long time since humans were capable of designing the circuits manually, given the feature size is now at the level where quantum mechanics must be accounted for or you get surprise tunnelling between gates; and (2) one of the things being automated is feature segmentation and labelling of neural micrographs, i.e. literally brain scanning: https://pubmed.ncbi.nlm.nih.gov/32619485/ https://pubmed.ncbi.nlm.nih.gov/32619485/
- nicomeemes 2y ago
  You're lazily mixing metaphors here. This is the problem in all such discussions, it often gets reduced to some combination of hand waving and hype training. "AGI" means different things to everyone, okay? Then it's a meaningless term. It's like saying hey with a quantum computer of enormous size, we could simulate molecular interactions at a level impossible with current technology. I would love for us to be able to do that- but where is the evidence it is even possible?
  - ben_w 2y ago
    There's no metaphor here, I meant literally doing those things. I followed up the point about AGI meaning different things by giving a common and sufficient standard of reference. Your brain is evidence that it's "even possible".
    - namaria 2y ago
      > Your brain is evidence that it's "even possible". All your brain proves is that a universe can produce planetary ecosystems capable of supporting human civilizations made of very efficient brain carrying mammals. It definitely doesn't prove that these mammals can create boxes capable 'solve physics, poverty and global warming' if we just give Sam Altman enough electricity and chips. Or dollars to that effect.
      - ben_w 2y ago
        If that's what you meant, then I agree with you. What's the quote? "If the human brain were so simple that we could understand it, we would be so simple that we couldn’t". Even though it doesn't need to be a single human doing all of it, our brains as existence proofs of the physical possibility, not of our own understanding.
        namaria 2y ago
        Like I said, our brain proves that the way to solve the problem of 'how to get autonomous systems capable of non trivially complex behavior' is 'planetary ecosystems sustained by starlight based on self reproducing cellular nanobots'. Companies selling ways around that apparatus to achieve intelligent behavior are selling you perpetual motion machines, or slavery. There has been zero 'AI' systems that don't depend on painstakingly collected and analyzed data beforehand.

ben_w 2y ago

> It's physically impossible Your brain is proof to the contrary. AGI means different things to everyone, but a human brain definitely counts as "general intelligence", that implemented in silicon is enough to get basically all the things promised by AGI: if that's done at the 20 watts per brain that biology manages, then all of humanity can be simulated within the power envelope of the USA electrical grid… three times over.

namaria 2y ago
> a human brain definitely counts as "general intelligence", that implemented in silicon is enough to get basically all the things promised by AGI: if that's done at the 20 watts per brain that biology manages, then all of humanity can be simulated within the power envelope of the USA electrical grid… three times over. So far the only thing that has been proven is we can get low entropy from all the low entropy we've published on the internet. Will it get to a point where models can give us more low entropy than what is present in the training data? Categorically: no.
- ben_w 2y ago
  You are using "entropy" in a way I do not recognise. Whatever you mean, our brains prove it's possible to have a system that uses 20 watts to demonstrate human-level intelligence.
  - namaria 2y ago
    Whatever you mean, it took 13 billion years and the whole universe as far as we can tell to create the 20 watt human intelligence brain. And it doesn't take 20 watt of energy to maintain humans, it takes a whole ecosystem capable of sustaining a human population. And for us to operate at our current level of information processing it takes a whole planetary civilization. So no you haven't proved anything by saying that a human brain consumes only 20 watts of energy. We spread an awful lot of entropy around that has to be kept away from our delicate biological and social systems. You're positing a way to create human intelligence-like in a bottle, that's the same as speculating about the shape of a reality where we have FTL travel or teleportation or whatever else you fancy. If we're talking about what current ML/AI can do, they can extract patterns from training data and than apply those patterns to other inputs. This can give us great automation, but it won't give us anything better than the training data, solve physics, global warming, poverty or give us human intelligence in a chip. Whatever quantity Q of entropy in the training data, the total output will be more Q all accounted for. That's true for humans and machines. No shape of possible AGI will give us any output with less Q than the combination of inputs had. The dream that a machine will solve all problems that humanity can't hinges on negating that, which goes against thermodynamics. As it stands, the planet cannot cope with all the entropy we're spreading around. It will eventually collapse civilization/the ecosystem, whatever buckles first, from the excess entropy. Because global warming, poverty, or ignorance is just entropy. Disorder. Things not being just so as we need them to be.
    - ben_w 2y ago
      > Whatever you mean, it took 13 billion years and the whole universe as far as we can tell to create the 20 watt human intelligence brain. It's grossly unreasonable to include the entire history of the universe given the Earth only formed about 4 billion years ago; and given that evolution wasn't even aiming for intelligence, even starting from our common ancestors being small rodents 65 million years ago is wildly overstating the effort required — even the evolution of primates is too far back without intentional selective breeding. > You're positing a way to create human intelligence-like in a bottle, that's the same as speculating about the shape of a reality where we have FTL travel or teleportation or whatever else you fancy. FTL may well well be impossible. If you seriously think human intelligence is impossible, then you also think you don't exist: You, yourself, are a human like intelligence in a bottle. The bottle being your skull. > This can give us great automation, but it won't solve physics, global warming, poverty or give us human intelligence in a chip. AI has already been in widesprad use in physics for a while now, well before the current zeitgeist of LLMs. There's a Y Combinator startup: "Charge Robotics is building robots that automate the most labor-intensive parts of solar construction". Poverty has many causes, some of which are already being reduced or resolved by existing systems — and that's been the case since one of the ancestor companies of IBM, Tabulating Machine Company, was doing punched cards for the US census. As for human intelligence on a chip? Well, (1) it's been quite a long time since humans were capable of designing the circuits manually, given the feature size is now at the level where quantum mechanics must be accounted for or you get surprise tunnelling between gates; and (2) one of the things being automated is feature segmentation and labelling of neural micrographs, i.e. literally brain scanning: https://pubmed.ncbi.nlm.nih.gov/32619485/ https://pubmed.ncbi.nlm.nih.gov/32619485/
nicomeemes 2y ago
You're lazily mixing metaphors here. This is the problem in all such discussions, it often gets reduced to some combination of hand waving and hype training. "AGI" means different things to everyone, okay? Then it's a meaningless term. It's like saying hey with a quantum computer of enormous size, we could simulate molecular interactions at a level impossible with current technology. I would love for us to be able to do that- but where is the evidence it is even possible?
- ben_w 2y ago
  There's no metaphor here, I meant literally doing those things. I followed up the point about AGI meaning different things by giving a common and sufficient standard of reference. Your brain is evidence that it's "even possible".
  - namaria 2y ago
    > Your brain is evidence that it's "even possible". All your brain proves is that a universe can produce planetary ecosystems capable of supporting human civilizations made of very efficient brain carrying mammals. It definitely doesn't prove that these mammals can create boxes capable 'solve physics, poverty and global warming' if we just give Sam Altman enough electricity and chips. Or dollars to that effect.
    - ben_w 2y ago
      If that's what you meant, then I agree with you. What's the quote? "If the human brain were so simple that we could understand it, we would be so simple that we couldn’t". Even though it doesn't need to be a single human doing all of it, our brains as existence proofs of the physical possibility, not of our own understanding.
      - namaria 2y ago
        Like I said, our brain proves that the way to solve the problem of 'how to get autonomous systems capable of non trivially complex behavior' is 'planetary ecosystems sustained by starlight based on self reproducing cellular nanobots'. Companies selling ways around that apparatus to achieve intelligent behavior are selling you perpetual motion machines, or slavery. There has been zero 'AI' systems that don't depend on painstakingly collected and analyzed data beforehand.