3 ms·

> Assumption 3 is mostly techno-optimism, or an extrapolation of the trend that we are able to build ever more advanced devices. This is your weak link. I don'

> Assumption 3 is mostly techno-optimism, or an extrapolation of the trend that we are able to build ever more advanced devices. This is your weak link. I don't see why progress will be a straight line and not a sloping-off curve. You shouldn't see the progress we've made in vehicle speed and assume we can hit the speed of light.

trashtester 2y ago
> This is your weak link. I don't think so at all. I'm personally convinced that humanity EVENTUALLY will build something more "intelligent" than human brains. > I don't see I see > You shouldn't see the progress we've made in vehicle speed and assume we can hit the speed of light There are laws of Physics that prevent us from moving faster than the speed of light. There IS a corresponding limit for computation [1], but it's about as far from the human brain's ability as the speed of light is from human running speed. I'm sure some people who saw the first cars thought they could never possibly become faster than a horse. Making ASI has no more reason to be impossible than to build something faster than the fastest animal, or (by stretching it), something faster than the speed of sound (which was supposed to be impossible). There is simply no reason to think that the human brain is at a global maximum when it comes to intelligence. Evolutionary history points towards brain size being limited by a combination of what is safe for the female hip width and also what amount of energy cost can be justified by increasing the size of the brain. Those who really think that humans have reached the peak, like David Deutsch, tend to think that the brain operates as a Turing Machine. And while a human brain CAN act like a very underpowered Turing Machine if given huge/infinite amounts of paper and time, that's not how most of our actual thought process function in reality. Since our ACTUAL thinking generally does NOT use Turing Complete computational facilities but rather relies on most information being stored in the actual neural net, the size of that net is a limiting factor for what mental operations a human can perform. I would claim that ONE way to create an object significantly more intelligent than current humans would be through genetic manipulation that would produce a "human" with a neocortex several times the size of what regular humans have. [1] https://en.wikipedia.org/wiki/Quantum_speed_limit https://en.wikipedia.org/wiki/Quantum_speed_limit
- czl 2y ago
  > Evolutionary history points towards brain size being limited by a combination of what is safe for the female hip width and also what amount of energy cost can be justified by increasing the size of the brain. If bigger brains lead to higher intelligence, why do many highly intelligent people have average-sized heads? And do they need to eat much more to fuel their high IQs? If larger brains were always better, wouldn’t female hips have evolved to accommodate them? I think human IQ might be where it is because extremely high intelligence vs. what we on average have now) often leads to fewer descendants. Less awareness of reality can lead to more "reproductive bliss."
  - trashtester 2y ago
    > If bigger brains lead to higher intelligence, why do many highly intelligent people have average-sized heads? There IS a correlation between intelligence and brain size (of about 0.3). But the human brain does a lot of things apart from what we measure as "IQ". What shows up in IQ tests tend to be mostly related to variation of the thickness of certain areas of the cortex [1]. The rest of the brain is, however, responsible from a lot of the functions that separates GPT-4 or Tesla's self driving from a human. Those are things we tend to take for granted in healthy humans, or that can show up as talents we don't think of as "intelligence". Also, the variation in the size of human brains is relatively small, so the specifics of how a given brain is organized probably contributes to more of the total variance than absolute size. That being said, a chimp brain is not likely to produce (adult, healthy) human level intelligence. > And do they need to eat much more to fuel their high IQs? That depends on the size of the brain, primarily. Human brains consume significantly more calories than a chimp brains. > If larger brains were always better, wouldn’t female hips have evolved to accommodate them? They did, and significantly so. In particular the part around the birth canal. > I think human IQ might be where it is because extremely high intelligence vs. what we on average have now) often leads to fewer descendants. Less awareness of reality can lead to more "reproductive bliss." I believe this is more of a modern phenomenon, mostly affecting women from the 20th century on. There may have been similar situations at times in the past, too. But generally, over the last several million years, human intelligence has been rising sharply. [1] https://www.sciencedirect.com/science/article/abs/pii/S0306452218304834#:~:text=Stronger%20IQ%20abilities%20are%20associated,beyond%20fronto%2Dparietal%20brain%20regions https://www.sciencedirect.com/science/article/abs/pii/S03064....
czl 2y ago
> I don't see why progress will be a straight line and not a sloping-off curve. Technological progress often appears linear in the short term, but zooming out reveals an exponential curve, similar to compound interest. > You shouldn't see the progress we've made in vehicle speed and assume we can hit the speed of light. Consider the trajectory of maximum speeds over millennia, not just recent history. We've achieved speeds unimaginable to our ancestors, mostly in space—a realm they couldn't conceive. While reaching light speed is challenging, we're exploring novel concepts like light-propelled nano-vehicles. If consciousness is information-based, could light itself become a "vehicle"?
- robertlagrant 2y ago
  > While reaching light speed is challenging, we're exploring novel concepts like light-propelled nano-vehicles Do you think hitting light speed is an engineering problem or a fundamental constraints problem?
  - czl 2y ago
    Reaching light speed isn't just an engineering problem—it's a fundamental issue in physics. The laws of physics (as we know them) prevent any object with mass from reaching light speed. Notice however that our minds, like the instructions in DNA and RNA, are built from atoms, but they aren't the atoms themselves. They're the information in how those atoms are arranged. Once we can fully read and write this information—like we're starting to do with DNA and RNA—light itself could become our vehicle.
  - trashtester 2y ago
    If even a single electron moves at the speed of light, it would tear apart the universe, at least that's what both special and general relativity would predict. (It would have infinite energy meaning infinite relativistic mass, and would form a black hole whose event horizon would spread into space at the speed of light).
    - robertlagrant 2y ago

trashtester 2y ago

> This is your weak link. I don't think so at all. I'm personally convinced that humanity EVENTUALLY will build something more "intelligent" than human brains. > I don't see I see > You shouldn't see the progress we've made in vehicle speed and assume we can hit the speed of light There are laws of Physics that prevent us from moving faster than the speed of light. There IS a corresponding limit for computation [1], but it's about as far from the human brain's ability as the speed of light is from human running speed. I'm sure some people who saw the first cars thought they could never possibly become faster than a horse. Making ASI has no more reason to be impossible than to build something faster than the fastest animal, or (by stretching it), something faster than the speed of sound (which was supposed to be impossible). There is simply no reason to think that the human brain is at a global maximum when it comes to intelligence. Evolutionary history points towards brain size being limited by a combination of what is safe for the female hip width and also what amount of energy cost can be justified by increasing the size of the brain. Those who really think that humans have reached the peak, like David Deutsch, tend to think that the brain operates as a Turing Machine. And while a human brain CAN act like a very underpowered Turing Machine if given huge/infinite amounts of paper and time, that's not how most of our actual thought process function in reality. Since our ACTUAL thinking generally does NOT use Turing Complete computational facilities but rather relies on most information being stored in the actual neural net, the size of that net is a limiting factor for what mental operations a human can perform. I would claim that ONE way to create an object significantly more intelligent than current humans would be through genetic manipulation that would produce a "human" with a neocortex several times the size of what regular humans have. [1] https://en.wikipedia.org/wiki/Quantum_speed_limit https://en.wikipedia.org/wiki/Quantum_speed_limit

czl 2y ago
> Evolutionary history points towards brain size being limited by a combination of what is safe for the female hip width and also what amount of energy cost can be justified by increasing the size of the brain. If bigger brains lead to higher intelligence, why do many highly intelligent people have average-sized heads? And do they need to eat much more to fuel their high IQs? If larger brains were always better, wouldn’t female hips have evolved to accommodate them? I think human IQ might be where it is because extremely high intelligence vs. what we on average have now) often leads to fewer descendants. Less awareness of reality can lead to more "reproductive bliss."
- trashtester 2y ago
  > If bigger brains lead to higher intelligence, why do many highly intelligent people have average-sized heads? There IS a correlation between intelligence and brain size (of about 0.3). But the human brain does a lot of things apart from what we measure as "IQ". What shows up in IQ tests tend to be mostly related to variation of the thickness of certain areas of the cortex [1]. The rest of the brain is, however, responsible from a lot of the functions that separates GPT-4 or Tesla's self driving from a human. Those are things we tend to take for granted in healthy humans, or that can show up as talents we don't think of as "intelligence". Also, the variation in the size of human brains is relatively small, so the specifics of how a given brain is organized probably contributes to more of the total variance than absolute size. That being said, a chimp brain is not likely to produce (adult, healthy) human level intelligence. > And do they need to eat much more to fuel their high IQs? That depends on the size of the brain, primarily. Human brains consume significantly more calories than a chimp brains. > If larger brains were always better, wouldn’t female hips have evolved to accommodate them? They did, and significantly so. In particular the part around the birth canal. > I think human IQ might be where it is because extremely high intelligence vs. what we on average have now) often leads to fewer descendants. Less awareness of reality can lead to more "reproductive bliss." I believe this is more of a modern phenomenon, mostly affecting women from the 20th century on. There may have been similar situations at times in the past, too. But generally, over the last several million years, human intelligence has been rising sharply. [1] https://www.sciencedirect.com/science/article/abs/pii/S0306452218304834#:~:text=Stronger%20IQ%20abilities%20are%20associated,beyond%20fronto%2Dparietal%20brain%20regions https://www.sciencedirect.com/science/article/abs/pii/S03064....

czl 2y ago

> I don't see why progress will be a straight line and not a sloping-off curve. Technological progress often appears linear in the short term, but zooming out reveals an exponential curve, similar to compound interest. > You shouldn't see the progress we've made in vehicle speed and assume we can hit the speed of light. Consider the trajectory of maximum speeds over millennia, not just recent history. We've achieved speeds unimaginable to our ancestors, mostly in space—a realm they couldn't conceive. While reaching light speed is challenging, we're exploring novel concepts like light-propelled nano-vehicles. If consciousness is information-based, could light itself become a "vehicle"?

robertlagrant 2y ago
> While reaching light speed is challenging, we're exploring novel concepts like light-propelled nano-vehicles Do you think hitting light speed is an engineering problem or a fundamental constraints problem?
- czl 2y ago
  Reaching light speed isn't just an engineering problem—it's a fundamental issue in physics. The laws of physics (as we know them) prevent any object with mass from reaching light speed. Notice however that our minds, like the instructions in DNA and RNA, are built from atoms, but they aren't the atoms themselves. They're the information in how those atoms are arranged. Once we can fully read and write this information—like we're starting to do with DNA and RNA—light itself could become our vehicle.
- trashtester 2y ago
  If even a single electron moves at the speed of light, it would tear apart the universe, at least that's what both special and general relativity would predict. (It would have infinite energy meaning infinite relativistic mass, and would form a black hole whose event horizon would spread into space at the speed of light).
  - robertlagrant 2y ago