3 ms·

Why would it not be bottlenecked? Is intelligence so valuable as such?

ben_w 3mo ago
Our intelligence was enough to turn some balding apes into atom-bomb wielders and astronauts over a few thousand generations. Consider a hypothetical: it is possible to make a semiconductor clone of a human brain, that runs at semiconductor speeds, has semiconductor size-scale, and has a power requirement of the Landauer limit. This copy would be smaller than a pea, and you'd get to pick on a sliding scale between "same power draw as human but thinks faster than us to the same multiplier that we jog faster than continental drift" or "thinks as fast as we do while using around a few µW of power". We do not know how to do this. We don't know how far we are from figuring out how to do this. We do know* evolution made our brains, and we do use simulated evolution as a standard technique in machine learning. But it may well be that just as no human knows how to design a human mind, we find the best AI we can make don't know how to make a better AI, at which point they're stumbling blind in the dark: while evolution is a neat method, it is limited, blind to what the best next step is at any moment. * well, those of us who are not creationists, at least.
- RandomLensman 3mo ago
  Is it possible to clone without destroying the original brain (and is a semiconductor the right substrate for it)? Thinking of the no cloning theorem here...
  - ben_w 3mo ago
    The no cloning theorem applies to quantum states; when it comes to information processed by it, it's unlikely that a big wet messy hot thing like a cell is usefully treated as "a quantum state", let alone an entire brain. However, this is irrelevant to the thought experiment, for which it is sufficient to merely be as competent as a human, rather than a completely quantum-perfect replica. Clone in the sense you would use the photoshop brush, or perhaps in the sense you might be sued for trademark infringement.
    - RandomLensman 3mo ago
      What is the thought experiment then to show other than a hypothetical piece of compute hardware? What's the experiment and its implication beyond the existence and its properties?
      - ben_w 3mo ago
        To show that RSI can, if bottlenecked like a human in quality, still represent a wild change from us for other reasons besides quality.
- defmacr0 3mo ago
  Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something like it happen. >we do use simulated evolution as a standard technique in machine learning. Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.
  - ben_w 3mo ago
    > Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. These are not the only currently-in-use AI models. However, recent news has even this particular category of AI solving multiple previously unsolved Erdős problems. (Even if they also do stupid things on a frequent basis). > Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications. I said "standard technique" rather than "best" for a reason ;) This is proof-of-possibility: natural selection did it, we know how to mimic that, but we don't know enough to be sure we're doing it with a reward function that will actually give us minds like ours on an interesting timescale with a probability high enough to care about.
    - godelski 3mo ago
      > solving multiple previously unsolved Erdős problems. So what? Machines have solved tons of unsolved problems in mathematics. That's not a proof of intelligence. If you brute force a solution, we congratulate you on your effort. If you stumble into a solution, we congratulate you for being lucky (if we can distinguish) If you find a unique solution that no one else imagined, we congratulate you on your intelligence. These are categorically different things and the difference matters. It's the whole distinction. Though in the real world success usually requires all three (and more), complicating evaluation. When we're talking about intelligence you can't distill it to "getting the answer". If you do then I'll direct you at an abacus, a calculator, a watch, or google search if you want to look at super intelligence. > natural selection did it Did what? Gradient descent? If that's the argument, you need to read more

ben_w 3mo ago

Our intelligence was enough to turn some balding apes into atom-bomb wielders and astronauts over a few thousand generations. Consider a hypothetical: it is possible to make a semiconductor clone of a human brain, that runs at semiconductor speeds, has semiconductor size-scale, and has a power requirement of the Landauer limit. This copy would be smaller than a pea, and you'd get to pick on a sliding scale between "same power draw as human but thinks faster than us to the same multiplier that we jog faster than continental drift" or "thinks as fast as we do while using around a few µW of power". We do not know how to do this. We don't know how far we are from figuring out how to do this. We do know* evolution made our brains, and we do use simulated evolution as a standard technique in machine learning. But it may well be that just as no human knows how to design a human mind, we find the best AI we can make don't know how to make a better AI, at which point they're stumbling blind in the dark: while evolution is a neat method, it is limited, blind to what the best next step is at any moment. * well, those of us who are not creationists, at least.

RandomLensman 3mo ago
Is it possible to clone without destroying the original brain (and is a semiconductor the right substrate for it)? Thinking of the no cloning theorem here...
- ben_w 3mo ago
  The no cloning theorem applies to quantum states; when it comes to information processed by it, it's unlikely that a big wet messy hot thing like a cell is usefully treated as "a quantum state", let alone an entire brain. However, this is irrelevant to the thought experiment, for which it is sufficient to merely be as competent as a human, rather than a completely quantum-perfect replica. Clone in the sense you would use the photoshop brush, or perhaps in the sense you might be sued for trademark infringement.
  - RandomLensman 3mo ago
    What is the thought experiment then to show other than a hypothetical piece of compute hardware? What's the experiment and its implication beyond the existence and its properties?
    - ben_w 3mo ago
      To show that RSI can, if bottlenecked like a human in quality, still represent a wild change from us for other reasons besides quality.
defmacr0 3mo ago
Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something like it happen. >we do use simulated evolution as a standard technique in machine learning. Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.
- ben_w 3mo ago
  > Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. These are not the only currently-in-use AI models. However, recent news has even this particular category of AI solving multiple previously unsolved Erdős problems. (Even if they also do stupid things on a frequent basis). > Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications. I said "standard technique" rather than "best" for a reason ;) This is proof-of-possibility: natural selection did it, we know how to mimic that, but we don't know enough to be sure we're doing it with a reward function that will actually give us minds like ours on an interesting timescale with a probability high enough to care about.
  - godelski 3mo ago
    > solving multiple previously unsolved Erdős problems. So what? Machines have solved tons of unsolved problems in mathematics. That's not a proof of intelligence. If you brute force a solution, we congratulate you on your effort. If you stumble into a solution, we congratulate you for being lucky (if we can distinguish) If you find a unique solution that no one else imagined, we congratulate you on your intelligence. These are categorically different things and the difference matters. It's the whole distinction. Though in the real world success usually requires all three (and more), complicating evaluation. When we're talking about intelligence you can't distill it to "getting the answer". If you do then I'll direct you at an abacus, a calculator, a watch, or google search if you want to look at super intelligence. > natural selection did it Did what? Gradient descent? If that's the argument, you need to read more