4 ms·

As a professional programmer and a relatively optimistic AGI enthusiast, why would the current ML methods not work, given sufficient CPU/RAM/GPU/latency/bandwid

As a professional programmer and a relatively optimistic AGI enthusiast, why would the current ML methods not work, given sufficient CPU/RAM/GPU/latency/bandwidth/storage? In theory, as long as you can translate your inputs and outputs to an array of floats, a neural network can compute anything. The required number of neurons might not fit into the world's best RAM, and the required number of weights and biases for those neurons might not be quickly calculated by a CPU/GPU however.

azinman2 4y ago
Yes but that doesn’t mean you won’t need new architectures or training methods to get there, or data that doesn’t currently exist. We also don’t know how many neurons / layers we’d need, etc. The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.
- readonlybarbie 4y ago
  This is an interesting comment. I agree that I hear the "all we need is 86 billion neurons and we will habe parity with the human brain", and I feel it is dubious to think this way because there is no reason why this arbitrary number must work. I also think it is a bit strange to use the human brain as an analogy because biological neurons supposedly are booleans and act in groups to achieve float level behavior. For example I can have neurologic pain in my fingers that isn't on off, but rather, has differences in magnitude. I think we should move away from the biology comparisons and just seek to understand if "more neurons = more better" is true, and if it is, how do we shove more into RAM and handle the exploding compute complexity.
  - eternalban 4y ago
    Watch this: Insights into AI algorithms drawn from hippocampal function: https://youtu.be/7yzcarJFibc https://youtu.be/7yzcarJFibc
jiggawatts 4y ago
The current AI approach is like a pure function in programming: no side effects, and given the same input you always get the same output. The “usage” and “training” steps are seperate. There is no episodic memory, especially there is no short term memory. Biological networks that result in conscious “minds” have a ton of loops and are constantly learning. You can essentially cut yourself off from the outside world in something like a sensory deprivation bath and your mind will continue to operate, talking to itself. No current popular and successful AI/ML approach can do anything like this.
- readonlybarbie 4y ago
  Agreed, but I also wonder if this is a "necessary" requirement. A robot, perhaps pretrained in a highly accurate 3d physics virtual simulation, which has an understanding of how it can move itself and others in the world, and how to accomplish text defined tasks, is already extremely useful and much more general than an image classificiation system. It is so general, in fact, that it would begin reliably replacing jobs.
  - jimbokun 4y ago
    But it's not AGI.
    - readonlybarbie 4y ago
      Ok, so now we just have to define "AGI" then. A robot, which knows its physical capabilities, which can see the world around it through a frustrum and identifies objects by position, velocity, rotation, which understands the passage of time and can predict future positions for example, which can take text input and translate that into a list of steps it needs to execute, which is functionally equivalent to an Amazon warehouse employee, we are saying is not AGI. What is an AGI then?
      - phphphphp 4y ago
        An Amazon warehouse worker isn’t a human, an Amazon warehouse worker is a human engaged in an activity that utilises a tiny portion of what that human is capable of. A Roomba is not AGI because it can do what a cleaner does. “Artificial general intelligence (AGI) is the ability of an intelligent agent to understand or learn any intellectual task that a human being can.”
tmoertel 4y ago
One big gap is causal learning. A true general intelligence will have to learn how to intervene in the real world to cause wanted outcomes in novel scenarios. Most current ML models capture only stastical knowledge. They can tell you what interventions have been associated with wanted outcomes in the past. In some situations, replaying these associations seems like genuine causal knowledge, but in novel scenarios this falls short. Even in current day models designed to make causal inferences, say for autonomous driving, the causal structure is more likely to have been built into the models by humans, rather than inferred from observations.
dinkumthinkum 4y ago
Well, I would put it back like why would it? When you understand how these things work, does it sound anything like what humans do? When prompted with a question, we do not respond by predicting words that come next based on a gigantic corpus of pre-trained text. As a professional programmer, do you think Human intelligence works like a Turing machine?
- readonlybarbie 4y ago
  The first interesting thing I'm told is that real biological neurons operate as booleans in the real world, whereas in computer land I'm told it is preferable to use float neurons. I suppose that you could chain biology neurons together in groups to achieve float-like behavior. So that's just one small example that we don't need AGI to be a model of the real human brain, with synapses and blood-brain barriers and everything. Rather, we just need one system to do n number of tasks at roughly the same level as a human for it to be "general". Maybe it's not AGI, but it also is not a hardcoded robotic arm that can only work with square objects of a certain dimension. If you had a robot that was pretrained in a virtual world, assembled in the real world, and then it begins testing and observing and resolving its own physical capabilities (moving arms and legs to stand and jump and backflip)... and then it also had a vision system to scan for threats and objectives... and then it also could resolve text and voice prompts to learn its next objective ("go get my favorite beer can from the fridge")... and the robot knows to ask you more questions to learn what your favorite beer is and also it knows how to preserve its own life in case the dog attacks it or the fridge topples over on it... then I think you have an extremely useful tool that will change the world, regardless of if it is labeled as AGI or not.

azinman2 4y ago

Yes but that doesn’t mean you won’t need new architectures or training methods to get there, or data that doesn’t currently exist. We also don’t know how many neurons / layers we’d need, etc. The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.

readonlybarbie 4y ago
This is an interesting comment. I agree that I hear the "all we need is 86 billion neurons and we will habe parity with the human brain", and I feel it is dubious to think this way because there is no reason why this arbitrary number must work. I also think it is a bit strange to use the human brain as an analogy because biological neurons supposedly are booleans and act in groups to achieve float level behavior. For example I can have neurologic pain in my fingers that isn't on off, but rather, has differences in magnitude. I think we should move away from the biology comparisons and just seek to understand if "more neurons = more better" is true, and if it is, how do we shove more into RAM and handle the exploding compute complexity.
- eternalban 4y ago
  Watch this: Insights into AI algorithms drawn from hippocampal function: https://youtu.be/7yzcarJFibc https://youtu.be/7yzcarJFibc

jiggawatts 4y ago

The current AI approach is like a pure function in programming: no side effects, and given the same input you always get the same output. The “usage” and “training” steps are seperate. There is no episodic memory, especially there is no short term memory. Biological networks that result in conscious “minds” have a ton of loops and are constantly learning. You can essentially cut yourself off from the outside world in something like a sensory deprivation bath and your mind will continue to operate, talking to itself. No current popular and successful AI/ML approach can do anything like this.

readonlybarbie 4y ago
Agreed, but I also wonder if this is a "necessary" requirement. A robot, perhaps pretrained in a highly accurate 3d physics virtual simulation, which has an understanding of how it can move itself and others in the world, and how to accomplish text defined tasks, is already extremely useful and much more general than an image classificiation system. It is so general, in fact, that it would begin reliably replacing jobs.
- jimbokun 4y ago
  But it's not AGI.
  - readonlybarbie 4y ago
    Ok, so now we just have to define "AGI" then. A robot, which knows its physical capabilities, which can see the world around it through a frustrum and identifies objects by position, velocity, rotation, which understands the passage of time and can predict future positions for example, which can take text input and translate that into a list of steps it needs to execute, which is functionally equivalent to an Amazon warehouse employee, we are saying is not AGI. What is an AGI then?
    - phphphphp 4y ago
      An Amazon warehouse worker isn’t a human, an Amazon warehouse worker is a human engaged in an activity that utilises a tiny portion of what that human is capable of. A Roomba is not AGI because it can do what a cleaner does. “Artificial general intelligence (AGI) is the ability of an intelligent agent to understand or learn any intellectual task that a human being can.”

tmoertel 4y ago

One big gap is causal learning. A true general intelligence will have to learn how to intervene in the real world to cause wanted outcomes in novel scenarios. Most current ML models capture only stastical knowledge. They can tell you what interventions have been associated with wanted outcomes in the past. In some situations, replaying these associations seems like genuine causal knowledge, but in novel scenarios this falls short. Even in current day models designed to make causal inferences, say for autonomous driving, the causal structure is more likely to have been built into the models by humans, rather than inferred from observations.

dinkumthinkum 4y ago

Well, I would put it back like why would it? When you understand how these things work, does it sound anything like what humans do? When prompted with a question, we do not respond by predicting words that come next based on a gigantic corpus of pre-trained text. As a professional programmer, do you think Human intelligence works like a Turing machine?

readonlybarbie 4y ago
The first interesting thing I'm told is that real biological neurons operate as booleans in the real world, whereas in computer land I'm told it is preferable to use float neurons. I suppose that you could chain biology neurons together in groups to achieve float-like behavior. So that's just one small example that we don't need AGI to be a model of the real human brain, with synapses and blood-brain barriers and everything. Rather, we just need one system to do n number of tasks at roughly the same level as a human for it to be "general". Maybe it's not AGI, but it also is not a hardcoded robotic arm that can only work with square objects of a certain dimension. If you had a robot that was pretrained in a virtual world, assembled in the real world, and then it begins testing and observing and resolving its own physical capabilities (moving arms and legs to stand and jump and backflip)... and then it also had a vision system to scan for threats and objectives... and then it also could resolve text and voice prompts to learn its next objective ("go get my favorite beer can from the fridge")... and the robot knows to ask you more questions to learn what your favorite beer is and also it knows how to preserve its own life in case the dog attacks it or the fridge topples over on it... then I think you have an extremely useful tool that will change the world, regardless of if it is labeled as AGI or not.