4 ms·

I think there are fundamental questions still unanswered that could put the brakes on the whole thing. For example, I would put forth that for a given problem,

I think there are fundamental questions still unanswered that could put the brakes on the whole thing. For example, I would put forth that for a given problem, there is a lower limit to how simple you can make divided pieces of that problem. You can't compute the trajectory of a thrown baseball using only a single transistor. Granted most problems can be divided into incredibly simple steps. The question we face is "is AGI reductive enough for human beings to create it?" Is the minimum work-unit for planning and constructing an AGI small enough to be understood by a human? That's of course putting aside the problem of scale. The neocortex alone has 160 trillion synapses, each of which exhibits behavior far more complex than a single transistor. You could argue that for many commercially-viable tasks we've found much better ways than nature, and that's true, but AGI is a different game entirely. Our current AI methodologies may be as unrelated to AGI as a spear is to a nuclear missile despite them both performing the same basic function.

beisner 6y ago
While I agree that designing and training a model the size of a human brain from scratch may well be an undertaking too great for any one human or even group of humans, that’s not how humans are created! I’d say the fact that we start life as a single cell with - at maximum - only a couple of gigabytes of information packed in our DNA and yet turn out to have profound intelligence is strong evidence that we might be able to design a system that - through some combination of obscenely efficient simple-ish learning algorithms - might be able to bootstrap its own intelligence in a similar fashion to how humans do. This isn’t necessarily a call for research to study how human embryos turn into thinking, feeling people, but more a loose upper bound on the complexity of the initial size of a model that could become AGI.
- computerphage 6y ago
  I agree with you, but it is also worth considering how much computation had to go into producing those couple of gigabytes.
lostmsu 6y ago
I think it is incorrect to say, that synapses exhibit more complex behavior in this context. How much of their behavior is actually part of cognition, as opposed to the needs of their own survival?
- jugg1es 6y ago
  The synapse isn't the complex part of the brain. The synapse is just the part of the neuron membrane that is responsible for input and output. The complex part is how the synapse affects potentiality inside the neuron itself. The difference between a transistor and a neuron is that a transistor has discrete, digital states. A neuron is much less deterministic than that. If you want to try to find a part of the brain that may represent a transistor, it would be a network of a number of specific neurons that fire (or just as importantly, dont fire) within an even larger superstructure. Ultimately, you just can't draw a clean line between a transistor and the brain at all.
- causality0 6y ago
  That's where our ignorance comes into play. We have no idea. It may even be a spectrum where some functions are more important than others but almost everything has a degree of effect on cognition. Brains don't play by the same rules as computers.
gwern 6y ago
> You can't compute the trajectory of a thrown baseball using only a single transistor. I wouldn't be so sure about that. You can catch a ball by keeping a constant angle. That seems like something a one-parameter model or PID may be able to do. Dragonflies catch evading prey using just 16 neurons: https://www.pnas.org/content/110/2/696.full https://www.pnas.org/content/110/2/696.full You may need a lot of neurons to learn something, but the found solutions may be quite small... Something to think about.
- causality0 6y ago
  Dragonflies visually track prey using only sixteen neurons. That's a testament to how complex a single neuron is, not how easy the task is. Dragonfly eyes each have thousands of facets and sensory cells. How many transistors would we need to process that input, in multiple colors, and identify and track prey in real time? It's a heck of a lot more than sixteen.

beisner 6y ago

While I agree that designing and training a model the size of a human brain from scratch may well be an undertaking too great for any one human or even group of humans, that’s not how humans are created! I’d say the fact that we start life as a single cell with - at maximum - only a couple of gigabytes of information packed in our DNA and yet turn out to have profound intelligence is strong evidence that we might be able to design a system that - through some combination of obscenely efficient simple-ish learning algorithms - might be able to bootstrap its own intelligence in a similar fashion to how humans do. This isn’t necessarily a call for research to study how human embryos turn into thinking, feeling people, but more a loose upper bound on the complexity of the initial size of a model that could become AGI.

computerphage 6y ago
I agree with you, but it is also worth considering how much computation had to go into producing those couple of gigabytes.

lostmsu 6y ago

I think it is incorrect to say, that synapses exhibit more complex behavior in this context. How much of their behavior is actually part of cognition, as opposed to the needs of their own survival?

jugg1es 6y ago
The synapse isn't the complex part of the brain. The synapse is just the part of the neuron membrane that is responsible for input and output. The complex part is how the synapse affects potentiality inside the neuron itself. The difference between a transistor and a neuron is that a transistor has discrete, digital states. A neuron is much less deterministic than that. If you want to try to find a part of the brain that may represent a transistor, it would be a network of a number of specific neurons that fire (or just as importantly, dont fire) within an even larger superstructure. Ultimately, you just can't draw a clean line between a transistor and the brain at all.
causality0 6y ago
That's where our ignorance comes into play. We have no idea. It may even be a spectrum where some functions are more important than others but almost everything has a degree of effect on cognition. Brains don't play by the same rules as computers.

gwern 6y ago

> You can't compute the trajectory of a thrown baseball using only a single transistor. I wouldn't be so sure about that. You can catch a ball by keeping a constant angle. That seems like something a one-parameter model or PID may be able to do. Dragonflies catch evading prey using just 16 neurons: https://www.pnas.org/content/110/2/696.full https://www.pnas.org/content/110/2/696.full You may need a lot of neurons to learn something, but the found solutions may be quite small... Something to think about.

causality0 6y ago
Dragonflies visually track prey using only sixteen neurons. That's a testament to how complex a single neuron is, not how easy the task is. Dragonfly eyes each have thousands of facets and sensory cells. How many transistors would we need to process that input, in multiple colors, and identify and track prey in real time? It's a heck of a lot more than sixteen.