5 ms·

The problem is the hard limits of physics. We have an upper bound for the number of interacting elements required for intelligence in the form of number of syna

by causi 4y ago

The problem is the hard limits of physics. We have an upper bound for the number of interacting elements required for intelligence in the form of number of synapses in the human brain. That number is a thousand trillion. We don't know what the lower bound is. Neurons are ferociously efficient computing devices, though. A dragonfly can take input from thousands of ommatidia and use it to track prey in three dimensions using only sixteen neurons. How many transistors do you need to do that? Let's be generous and say only ten percent of the human brain is needed for sapience, or 8.6 billion neurons. 8.6 billion / 16 roughly equals five hundred million. Multiply your earlier transistor number by five hundred million. There's your lower bound for a sapient computer. There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever.

ben_w 4y ago
> There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. A computer with five times the mass, five times the volume, and five times the power requirements of the human brain, that can run an uploaded copy of a human brain at merely half the speed of a human brain, would still render Pluto and Mercury trivial to colonise with machine intelligences. And if it cost a $100k (inflation adjusted) to build, and lasted 80 years before you had to melt down the hardware and reforge from scratch, it would still be cheaper than Musk's target for a human to Mars.
- sokoloff 4y ago
  That’s for the computing power, not for the physical actuators which would be needed to provide power, other environmentals, and repairs.
  - ben_w 4y ago
    Those are the easy bits. We've already got all of those, because that's how we make the machines we already use even today, and we would need them in any scenario other than genetically engineering ourselves to eat the moon, not just the uploaded brains scenario.
    - sokoloff 4y ago
      Do you spot us all the supply chain that underlies today’s technology economy? Mere COVID disrupted that here; imagine what a few hundred million miles of space would do.
      - ben_w 4y ago
        We know what all that stuff is and how it works, making a fresh supply chain in a new place isn't handled by magical mystery cult of the gods… oh, Plutus, and Mercury are the gods of wealth, that's a coincidence.
        sokoloff 4y ago
        How a lot of that stuff works on Earth is using abundant fossil fuels for mining, refining, and fossil fuels and biological material as feedstocks. In particular, it’s not obvious how to make volumes of virgin steel from iron ore without coke. (Electric arc remelting is fine for recycling steel to steel.) Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well.
        ben_w 4y ago
        > In particular, it’s not obvious how to make volumes of virgin steel from iron ore without coke. https://www.researchgate.net/publication/322275580_Electrolysis_of_iron_in_a_molten_oxide_electrolyte https://www.researchgate.net/publication/322275580_Electroly... > Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well. If we really needed those things (why would we even want to use wood as a structural element on a space habitat?), we can build bioreactors from metal, glass, and water. Oil and gas do occur non-biologically e.g. Titan, but they can also be made from algae grown in a transparent tube exposed to sunlight and provided with the necessary minerals and CO2, which is trivial to make. IIRC the two limited resources if you needed biology and couldn't do it all with a clanking replicator are nitrogen and phosphate, everything else is easy to find basically everywhere.
drran 4y ago
AI shows signs of general intelligence (e.g. can generate proper text in generated pictures) after about 20 billion of parameters. IMHO, 1 trillion of parameters can be enough to simulate human-like behaviour after reading lot of text, watching of lot of video, and playing lot of games and robotic simulations. It will be able to understand commands, understand video and audio from cameras, will be able to control equipment and robots, e.g. to repair itself, to reach a goal.
- simiones 4y ago
  You're only looking at the parameters, when in fact such algorithms are also heavily limited by the amount of training data. It may well be that, even if you were right with your 1 trillion parameters for AGI, we might just not have enough training data to avoid under/over fitting. Note that I am extremely skeptical of your 1T parameters claim - there is much much more to AGI than natural-looking natural language processing + image recognition.
sullyj3 4y ago
A priori one would not expect humans to be near the physical limit of what's possible, purely based on the characteristics of the process that created us Natural selection is - random - blind - gradient descent is prone to getting stuck in local minima, has no foresight, no ability to go back to the drawing board, no "understanding" of what it's doing. - not even optimizing for intelligence, except instrumentally - to the extent that increasing intelligence interferes with survival, it has to be sacrificed - subject to hard constraints like the limits on size imposed by childbirth
- CuriouslyC 4y ago
  Animals have been rewarded for solving diverse environmental challenges for a long time. Brains in general are well optimized, just maybe not for the objective function you care about.
Balgair 4y ago
Few nit picks on the neuro side: The synapse is an incredibly complicated structure and there is a lot of 'computation' occurring within the butons. We're not very sure of all the processes that occur at this time. It's not just a 0/1 kinda thing. Also, our current computers aren't even running the hardware that a synapse is. The most analogous electrical structure to a synapse is a memristor, something we can't make at scale right now. The synapse is also not the only structure that causes computation to occur, many things effect the firing of a neuron and that modulation. Great lower bound calculation though. I'd say that's the right ballpark number.
- causi 4y ago
  Yeah I know, but the "computational power will keep increasing forever" religion has a strong following on HN who react badly when you point out the true scale of the problem.
adamsmith143 4y ago
Lot of problems with your analysis. One is that you assume the brain is the most efficient way to generate intelligence. There doesn't seem to be any reason to make that assumption and it may be that von Neumann architectures are thousands or milions of times more efficient at that task. Another is that Neurons are incredibly slow. Lets say you can make a machine only half as intelligent as a Human but that can think 1 Million times faster, this is even a lowball since a neuron can fire at like 1Hz and modern CPUs operate in the GHz range. >In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. Think this is a really bad take. If one assumes even a .1% improvement in "Machine Intelligence" per unit time then it is literally only a matter of time until you reach Human level and then pass it.
- causi 4y ago
  If one assumes even a .1% improvement in "Machine Intelligence" per unit time then it is literally only a matter of time until you reach Human level and then pass it. That's like saying "even if there's only a 0.1% improvement in materials strength per unit time eventually we'll have structures strong enough to bounce off an incoming rogue planet." There is an upper bound to computational performance per unit mass and unit energy. We're nowhere near it but it is there, lurking.
  - adamsmith143 4y ago
    The problem is assuming the upper bound is close or at human level for intelligence. There's no reason to think that and clearly outlier humans like John von Neumann that are still in the Human range are far more capable than the average person. If you only could make machines as intelligent as that it would still be revolutionary.
- sullyj3 4y ago
  > a neuron can fire at like 1Hz and modern CPUs operate in the GHz range. This comparison doesn't make much sense - computers have a very small number of cores, let's say 10, and brains have 86 billion neurons. 86 billion things operating at 1Hz is also in the GHz range. This is leaving aside the issue that a CPU cycle and a neuron firing are doing completely different sorts of work - comparing them is kind of nonsensical in the first place.

by causi 4y ago

ben_w 4y ago

> There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. A computer with five times the mass, five times the volume, and five times the power requirements of the human brain, that can run an uploaded copy of a human brain at merely half the speed of a human brain, would still render Pluto and Mercury trivial to colonise with machine intelligences. And if it cost a $100k (inflation adjusted) to build, and lasted 80 years before you had to melt down the hardware and reforge from scratch, it would still be cheaper than Musk's target for a human to Mars.

sokoloff 4y ago
That’s for the computing power, not for the physical actuators which would be needed to provide power, other environmentals, and repairs.
- ben_w 4y ago
  Those are the easy bits. We've already got all of those, because that's how we make the machines we already use even today, and we would need them in any scenario other than genetically engineering ourselves to eat the moon, not just the uploaded brains scenario.
  - sokoloff 4y ago
    Do you spot us all the supply chain that underlies today’s technology economy? Mere COVID disrupted that here; imagine what a few hundred million miles of space would do.
    - ben_w 4y ago
      We know what all that stuff is and how it works, making a fresh supply chain in a new place isn't handled by magical mystery cult of the gods… oh, Plutus, and Mercury are the gods of wealth, that's a coincidence.
      - sokoloff 4y ago
        How a lot of that stuff works on Earth is using abundant fossil fuels for mining, refining, and fossil fuels and biological material as feedstocks. In particular, it’s not obvious how to make volumes of virgin steel from iron ore without coke. (Electric arc remelting is fine for recycling steel to steel.) Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well.
        ben_w 4y ago
        > In particular, it’s not obvious how to make volumes of virgin steel from iron ore without coke. https://www.researchgate.net/publication/322275580_Electrolysis_of_iron_in_a_molten_oxide_electrolyte https://www.researchgate.net/publication/322275580_Electroly... > Much of the plastic supply chain starts as oil or gas. Wood is a major building component as well. If we really needed those things (why would we even want to use wood as a structural element on a space habitat?), we can build bioreactors from metal, glass, and water. Oil and gas do occur non-biologically e.g. Titan, but they can also be made from algae grown in a transparent tube exposed to sunlight and provided with the necessary minerals and CO2, which is trivial to make. IIRC the two limited resources if you needed biology and couldn't do it all with a clanking replicator are nitrogen and phosphate, everything else is easy to find basically everywhere.

drran 4y ago

AI shows signs of general intelligence (e.g. can generate proper text in generated pictures) after about 20 billion of parameters. IMHO, 1 trillion of parameters can be enough to simulate human-like behaviour after reading lot of text, watching of lot of video, and playing lot of games and robotic simulations. It will be able to understand commands, understand video and audio from cameras, will be able to control equipment and robots, e.g. to repair itself, to reach a goal.

simiones 4y ago
You're only looking at the parameters, when in fact such algorithms are also heavily limited by the amount of training data. It may well be that, even if you were right with your 1 trillion parameters for AGI, we might just not have enough training data to avoid under/over fitting. Note that I am extremely skeptical of your 1T parameters claim - there is much much more to AGI than natural-looking natural language processing + image recognition.

sullyj3 4y ago

A priori one would not expect humans to be near the physical limit of what's possible, purely based on the characteristics of the process that created us Natural selection is - random - blind - gradient descent is prone to getting stuck in local minima, has no foresight, no ability to go back to the drawing board, no "understanding" of what it's doing. - not even optimizing for intelligence, except instrumentally - to the extent that increasing intelligence interferes with survival, it has to be sacrificed - subject to hard constraints like the limits on size imposed by childbirth

CuriouslyC 4y ago
Animals have been rewarded for solving diverse environmental challenges for a long time. Brains in general are well optimized, just maybe not for the objective function you care about.

Balgair 4y ago

Few nit picks on the neuro side: The synapse is an incredibly complicated structure and there is a lot of 'computation' occurring within the butons. We're not very sure of all the processes that occur at this time. It's not just a 0/1 kinda thing. Also, our current computers aren't even running the hardware that a synapse is. The most analogous electrical structure to a synapse is a memristor, something we can't make at scale right now. The synapse is also not the only structure that causes computation to occur, many things effect the firing of a neuron and that modulation. Great lower bound calculation though. I'd say that's the right ballpark number.

causi 4y ago
Yeah I know, but the "computational power will keep increasing forever" religion has a strong following on HN who react badly when you point out the true scale of the problem.

adamsmith143 4y ago

Lot of problems with your analysis. One is that you assume the brain is the most efficient way to generate intelligence. There doesn't seem to be any reason to make that assumption and it may be that von Neumann architectures are thousands or milions of times more efficient at that task. Another is that Neurons are incredibly slow. Lets say you can make a machine only half as intelligent as a Human but that can think 1 Million times faster, this is even a lowball since a neuron can fire at like 1Hz and modern CPUs operate in the GHz range. >In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. Think this is a really bad take. If one assumes even a .1% improvement in "Machine Intelligence" per unit time then it is literally only a matter of time until you reach Human level and then pass it.

causi 4y ago
If one assumes even a .1% improvement in "Machine Intelligence" per unit time then it is literally only a matter of time until you reach Human level and then pass it. That's like saying "even if there's only a 0.1% improvement in materials strength per unit time eventually we'll have structures strong enough to bounce off an incoming rogue planet." There is an upper bound to computational performance per unit mass and unit energy. We're nowhere near it but it is there, lurking.
- adamsmith143 4y ago
  The problem is assuming the upper bound is close or at human level for intelligence. There's no reason to think that and clearly outlier humans like John von Neumann that are still in the Human range are far more capable than the average person. If you only could make machines as intelligent as that it would still be revolutionary.
sullyj3 4y ago
> a neuron can fire at like 1Hz and modern CPUs operate in the GHz range. This comparison doesn't make much sense - computers have a very small number of cores, let's say 10, and brains have 86 billion neurons. 86 billion things operating at 1Hz is also in the GHz range. This is leaving aside the issue that a CPU cycle and a neuron firing are doing completely different sorts of work - comparing them is kind of nonsensical in the first place.