4 ms·

A really simple approach we took while I was working on a research team at Microsoft for predicting when AGI would land was simply estimating at what point can

by jjcm 3y ago

A really simple approach we took while I was working on a research team at Microsoft for predicting when AGI would land was simply estimating at what point can we run a full simulation of all of the chemical processes and synapses inside a human brain. The approach was tremendously simple and totally naive, but it was still interesting. At the time a supercomputer could simulate the full brain of a flatworm. We then simply applied a Moore's law-esque approach of assuming simulation capacity can double every 1.5-2 years (I forget the time period we used), and mapped out different animals that we had the capability to simulate on each date. We showed years for a field mouse, a corvid, a chimp, and eventually a human brain. The date we landed on was 2047. There are so many things wrong with that approach I can't even count, but I'd be kinda smitten if it ended up being correct.

tayo42 3y ago
Is there something to read about simulating a worm brain. Neurons aren't just simply on and off? They grow and adapt physically along with their chemical signals. Curious how a computer accounts for all of that.
- jncraton 3y ago
  You might be interested in OpenWorm: https://openworm.org/ https://openworm.org/ This paper might be helpful for understanding the nervous system in particular: https://royalsocietypublishing.org/doi/10.1098/rstb.2017.0379 https://royalsocietypublishing.org/doi/10.1098/rstb.2017.037...
  - deleted 3y ago
    [deleted]
Keloo 3y ago
How does it compare to the progress? Is the progress faster/slower? Any links to read?
shpongled 3y ago
To be pedantic, I would argue that we aren't even close to being able to simulate the full brain of a flatworm on a supercomputer at anything deeper than a simple representation of neurons. We can't even simulate all of the chemical processes inside a single cell. We don't even know all of the chemical processes. We don't know the function of most proteins.
- minroot 3y ago
  Does AGI must needs to be brain-like?
  - parl_match 3y ago
    No, but the simple approach here was "full simulation". And "brain in a jar" is different from "AGI"
  - dragonwriter 3y ago
    The human brain is the only thing we can conclusively say does run a general intelligence, so, its the level of complexity at which we can say confidently that its just a software/architecture problem. There may be (almost certainly is) a more optimized way a general intelligence could be implemented, but we can't confidentally say what that requires.
    - glial 3y ago
      > The human brain is the only thing we can conclusively say does run a general intelligence That's because we define "general intelligence" circularly as "something the human brain does."
      - Jensson 3y ago
        If something else could replace humanity at intellectual tasks we would say it is generally intelligent as well. Currently there is no such thing, we still need humanity to perform intellectual tasks.
throwup238 3y ago
Another approach: the adult human brain has 100 (+- 20) billion or 10^11 neurons. Each neuron has 10^3 synapses and each synapse has 10^2 ion channels, amounts to 10^16 total channels. Assuming 10 parameters is enough to represent each channel (unlikely), that's about 10^17 (100 quadrillion) total parameters. Compare that to GPT4 which is rumored to be about 1.7*10^12 parameters on 8x 80GB A100s. log(10^17/10^12)/log(2) = 16.61 so assuming 1.5 years per doubling, that'll be another 24.9 years - December, 2048 - before 8x X100s can simulate the human brain.
- WhitneyLand 3y ago
  And then how long until it runs on 20 watts of power? ;)
- redavni 3y ago
  > December, 2048 So it is not unreasonable to expect I can have an Ana de Armas AI in 2049? I hope you AI people are better than the flying car people.
daveguy 3y ago
I assume simulation capacity takes into account the data bandwidth of the processing systems. It seems we are always an orders of magnitude or two behind in bytes/words per second to feed simulations compared to raw flops. When you consider there are multiple orders of magnitude more synapses between neurons than neurons (not to mention other cell types we are only beginning to understand) -- bandwidth limitations seem to put estimates about 10-15 years past computation estimates. By my napkin math, accounting for bandwidth limitations, we will get single-human-intelligence hardware capabilities 2053-2063. Whether or not we've figured out the algorithms by then is any guess. Maybe algorithm advances will reduce hardware needs, but I doubt it because computational complexity to solve hard problems is often a matter of getting all the bits to the processor to perform all the comparisons necessary. However, the massive parallelism of the brain is a point of optimism.
ShamelessC 3y ago
> At the time a supercomputer could simulate the full brain of a flatworm. Citation needed?
- ShamelessC 3y ago
  Just going to follow up and say “I don’t think that statement is even remotely true now, much less back then”. We haven’t accurately simulated any life forms. Failure to simulate C. Elegans is notable.
reality71 3y ago
Your approach will eventually work, no doubt about it, but the question is whether the amount of energy the computer uses to complete a task is less than the energy the equivalent conglomeration of humans use to complete a task. It seems clear at this point that although computers can be made to model physical systems to great degree, this is not the area where they naturally excel. Think of modeling the temperature of a room, you could try and recreate the physically accurate simulation of every particle and its velocity. We could then create better software to model the particles on ever more powerful and specific hardware to model bigger and bigger rooms. Just like how thermodynamics might make more sense to model statistically, I think intelligence is not best modeled at the synapse layer. I think the much more interesting question is what would the equivalent of a worm brain be for a digital intelligence?

by jjcm 3y ago

tayo42 3y ago

Is there something to read about simulating a worm brain. Neurons aren't just simply on and off? They grow and adapt physically along with their chemical signals. Curious how a computer accounts for all of that.

jncraton 3y ago
You might be interested in OpenWorm: https://openworm.org/ https://openworm.org/ This paper might be helpful for understanding the nervous system in particular: https://royalsocietypublishing.org/doi/10.1098/rstb.2017.0379 https://royalsocietypublishing.org/doi/10.1098/rstb.2017.037...
- deleted 3y ago
  [deleted]

Keloo 3y ago

How does it compare to the progress? Is the progress faster/slower? Any links to read?

shpongled 3y ago

To be pedantic, I would argue that we aren't even close to being able to simulate the full brain of a flatworm on a supercomputer at anything deeper than a simple representation of neurons. We can't even simulate all of the chemical processes inside a single cell. We don't even know all of the chemical processes. We don't know the function of most proteins.

minroot 3y ago
Does AGI must needs to be brain-like?
- parl_match 3y ago
  No, but the simple approach here was "full simulation". And "brain in a jar" is different from "AGI"
- dragonwriter 3y ago
  The human brain is the only thing we can conclusively say does run a general intelligence, so, its the level of complexity at which we can say confidently that its just a software/architecture problem. There may be (almost certainly is) a more optimized way a general intelligence could be implemented, but we can't confidentally say what that requires.
  - glial 3y ago
    > The human brain is the only thing we can conclusively say does run a general intelligence That's because we define "general intelligence" circularly as "something the human brain does."
    - Jensson 3y ago
      If something else could replace humanity at intellectual tasks we would say it is generally intelligent as well. Currently there is no such thing, we still need humanity to perform intellectual tasks.

throwup238 3y ago

Another approach: the adult human brain has 100 (+- 20) billion or 10^11 neurons. Each neuron has 10^3 synapses and each synapse has 10^2 ion channels, amounts to 10^16 total channels. Assuming 10 parameters is enough to represent each channel (unlikely), that's about 10^17 (100 quadrillion) total parameters. Compare that to GPT4 which is rumored to be about 1.7*10^12 parameters on 8x 80GB A100s. log(10^17/10^12)/log(2) = 16.61 so assuming 1.5 years per doubling, that'll be another 24.9 years - December, 2048 - before 8x X100s can simulate the human brain.

WhitneyLand 3y ago
And then how long until it runs on 20 watts of power? ;)
redavni 3y ago
> December, 2048 So it is not unreasonable to expect I can have an Ana de Armas AI in 2049? I hope you AI people are better than the flying car people.

daveguy 3y ago

I assume simulation capacity takes into account the data bandwidth of the processing systems. It seems we are always an orders of magnitude or two behind in bytes/words per second to feed simulations compared to raw flops. When you consider there are multiple orders of magnitude more synapses between neurons than neurons (not to mention other cell types we are only beginning to understand) -- bandwidth limitations seem to put estimates about 10-15 years past computation estimates. By my napkin math, accounting for bandwidth limitations, we will get single-human-intelligence hardware capabilities 2053-2063. Whether or not we've figured out the algorithms by then is any guess. Maybe algorithm advances will reduce hardware needs, but I doubt it because computational complexity to solve hard problems is often a matter of getting all the bits to the processor to perform all the comparisons necessary. However, the massive parallelism of the brain is a point of optimism.

ShamelessC 3y ago

> At the time a supercomputer could simulate the full brain of a flatworm. Citation needed?

ShamelessC 3y ago
Just going to follow up and say “I don’t think that statement is even remotely true now, much less back then”. We haven’t accurately simulated any life forms. Failure to simulate C. Elegans is notable.

reality71 3y ago

Your approach will eventually work, no doubt about it, but the question is whether the amount of energy the computer uses to complete a task is less than the energy the equivalent conglomeration of humans use to complete a task. It seems clear at this point that although computers can be made to model physical systems to great degree, this is not the area where they naturally excel. Think of modeling the temperature of a room, you could try and recreate the physically accurate simulation of every particle and its velocity. We could then create better software to model the particles on ever more powerful and specific hardware to model bigger and bigger rooms. Just like how thermodynamics might make more sense to model statistically, I think intelligence is not best modeled at the synapse layer. I think the much more interesting question is what would the equivalent of a worm brain be for a digital intelligence?