3 ms·

Can you define 'near' for me? I think AGI is likely closer to the present than 1987 was -- that is, I'd bet on having AGI by 2047. (Note: this is distinct from

Can you define 'near' for me? I think AGI is likely closer to the present than 1987 was -- that is, I'd bet on having AGI by 2047. (Note: this is distinct from superhuman AGI.) Do you not agree? I think a lot of people underestimate NNs because they think of NNs in terms of the semantics of their history instead of all possible semantics that can be fit to tensor networks. We know [P] that NNs are a sufficient abstraction to model human intelligence if we had arbitrary compute -- the questions that remain are all about making the hardware faster enough and the estimators efficient enough (which may require moving off tensor networks, but it's still only a refinement of the mathematics used). Of course, one could argue that humans are caught in a "tensor trap", in that too much of our intellectual effort is now relying on estimators built out of networks of tensors. (I do.) But even then, AGI is likely to appear out of similar methods with new mathematical objects. [P] Proof NNs can compute human intelligence with arbitrary compute: You can embed the standard model as a NN by changing how you view the network of tensor equations. Human intelligence is (arguably) embeded in the standard model by modern science. So we can embed a model of human intelligence in a (large enough) NN. This isn't immediately computationally useful, but it shows that there's not a fundamental flaw in using an estimator built out of a DAG of calculations to model intelligence if we can find an appropriate estimator for our computational needs.

Figs 9y ago
> DAG Feedback and memory are really important features of GI that you will not get out of a DAG ever. You need loops for that.
- SomeStupidPoint 9y ago
  > estimator built out of a DAG of calculations All loops can be modeled as a DAG and single attached piece of memory (of sufficient width) allowed to execute to a steady state; sorry if it wasn't clear that I was talking about things like NTMs too. (It's why I used 'tensor network' most places; also, in practice, we tend to let subgraphs reach a steady state independently where possible.) Your comment is also an excellent example of a strawman: you picked out the word 'DAG' to raise a technical argument when the usage of DAG versus general tensor networks clearly wasn't the main point (as some NNs have feedback and the standard model is posed as differential equations). It's more constructive to respond to the strongest point, not pick at technical details that can easily be rephrased.
leereeves 9y ago
The ability to compute intelligence given infinite time and compute power isn't proof that NNs are a useful approach. For that, a technique must be able to compute human intelligence at a useful speed on a constructable system.
- SomeStupidPoint 9y ago
  Didn't the same argument apply in general to why NNs weren't particularly useful for anything ~30 years ago? Using a network of tensors to compute intelligence is incredibly old (I believe, dating back about 80 years), but has only recently become tractable to do for any complex tasks. However, in the past ~30 years, we've gone from "intractable for moderate problems" to "world champion at go", "able to detect cancer in images as well as experts", etc. My contention is that in another ~30 years, we'll see a step sufficient for "can do average at most intellectual activities", even if that's just having the storage to keep 10,000 task specific NNs (of AlphaGo sophistication) on hand to interpolate all actions as mixes of specialist tasks. Do you really not think there's a strong heuristic case for that? (I would contend that you should be able to point to a specific task you don't think it will be able to do on that timeline -- do you know of such a task?) The proof was merely that we're not barking up a theoretically dead tree -- we have to rely on heuristics for if it will eventually converge to tractable.
  - leereeves 9y ago
    I'm not arguing that NNs aren't capable of AGI, merely that the ability to compute the standard model leaves the far more difficult question of whether the problem is tractable, as you said. The standard model could be computed directly without NNs, which I think we agree wouldn't be a useful way to approach AGI.
  - mannykannot 9y ago
    > Didn't the same argument apply in general to why NNs weren't particularly useful for anything ~30 years ago? Not unless there was good reason to expect that every other possible application of NNs would be as difficult to achieve as general intelligence.
sago 9y ago
> Can you define 'near' for me? Not sensibly in terms of years, no. It's more a handwaving gut feeling combined with an intuition. I didn't feel like there was a royal road from symbolic AI to AGI. It doesn't feel to me like there is one from NNs either. As for the intuition: perhaps it's because it was my PhD topic, I have always felt that there needs to be a breakthrough in emergence, specifically in evolutionary computing (or some other system in which there is a tight feedback loop between behaviour and survival). Something to unshackle the development of AI from human beings deciding what behaviours they want to engineer. The resulting computation would be orders of magnitude less efficient, considerably less understandable (it is very unlikely to wash dishes or write poetry), but crucially much less fragile. And it has always been the fragility of the engineering which has made AI feel a little smoke and mirrors at times. NN are massively less fragile than symbolic systems, (and orders of magnitude less efficient, for problems symbolic systems are good at) but it does feel like we need another fundamental step. But, my feelings aside, I agree with the article because I recognise this could well be a 'Manhattan Project' type of event.
- eb3c90 9y ago
  My bet on this subject is a combination of evolutionary computing, agorics[1], deep learning style feedback and language learning. I've got a repo[2] for a vm where the programs can act in that way. But I am at the early stages of programming initial programs with economic/learning strategies. So I don't know how promising it is. More details can be found spread out on my blog [3] [1] https://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.html https://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.htm... [2] https://github.com/eb4890/agorint/ https://github.com/eb4890/agorint/ [3] https://improvingautonomy.wordpress.com/2017/08/13/introduction-to-agorint-part-1 https://improvingautonomy.wordpress.com/2017/08/13/introduct...

Figs 9y ago

> DAG Feedback and memory are really important features of GI that you will not get out of a DAG ever. You need loops for that.

SomeStupidPoint 9y ago
> estimator built out of a DAG of calculations All loops can be modeled as a DAG and single attached piece of memory (of sufficient width) allowed to execute to a steady state; sorry if it wasn't clear that I was talking about things like NTMs too. (It's why I used 'tensor network' most places; also, in practice, we tend to let subgraphs reach a steady state independently where possible.) Your comment is also an excellent example of a strawman: you picked out the word 'DAG' to raise a technical argument when the usage of DAG versus general tensor networks clearly wasn't the main point (as some NNs have feedback and the standard model is posed as differential equations). It's more constructive to respond to the strongest point, not pick at technical details that can easily be rephrased.

leereeves 9y ago

The ability to compute intelligence given infinite time and compute power isn't proof that NNs are a useful approach. For that, a technique must be able to compute human intelligence at a useful speed on a constructable system.

SomeStupidPoint 9y ago
Didn't the same argument apply in general to why NNs weren't particularly useful for anything ~30 years ago? Using a network of tensors to compute intelligence is incredibly old (I believe, dating back about 80 years), but has only recently become tractable to do for any complex tasks. However, in the past ~30 years, we've gone from "intractable for moderate problems" to "world champion at go", "able to detect cancer in images as well as experts", etc. My contention is that in another ~30 years, we'll see a step sufficient for "can do average at most intellectual activities", even if that's just having the storage to keep 10,000 task specific NNs (of AlphaGo sophistication) on hand to interpolate all actions as mixes of specialist tasks. Do you really not think there's a strong heuristic case for that? (I would contend that you should be able to point to a specific task you don't think it will be able to do on that timeline -- do you know of such a task?) The proof was merely that we're not barking up a theoretically dead tree -- we have to rely on heuristics for if it will eventually converge to tractable.
- leereeves 9y ago
  I'm not arguing that NNs aren't capable of AGI, merely that the ability to compute the standard model leaves the far more difficult question of whether the problem is tractable, as you said. The standard model could be computed directly without NNs, which I think we agree wouldn't be a useful way to approach AGI.
- mannykannot 9y ago
  > Didn't the same argument apply in general to why NNs weren't particularly useful for anything ~30 years ago? Not unless there was good reason to expect that every other possible application of NNs would be as difficult to achieve as general intelligence.

sago 9y ago

> Can you define 'near' for me? Not sensibly in terms of years, no. It's more a handwaving gut feeling combined with an intuition. I didn't feel like there was a royal road from symbolic AI to AGI. It doesn't feel to me like there is one from NNs either. As for the intuition: perhaps it's because it was my PhD topic, I have always felt that there needs to be a breakthrough in emergence, specifically in evolutionary computing (or some other system in which there is a tight feedback loop between behaviour and survival). Something to unshackle the development of AI from human beings deciding what behaviours they want to engineer. The resulting computation would be orders of magnitude less efficient, considerably less understandable (it is very unlikely to wash dishes or write poetry), but crucially much less fragile. And it has always been the fragility of the engineering which has made AI feel a little smoke and mirrors at times. NN are massively less fragile than symbolic systems, (and orders of magnitude less efficient, for problems symbolic systems are good at) but it does feel like we need another fundamental step. But, my feelings aside, I agree with the article because I recognise this could well be a 'Manhattan Project' type of event.

eb3c90 9y ago
My bet on this subject is a combination of evolutionary computing, agorics[1], deep learning style feedback and language learning. I've got a repo[2] for a vm where the programs can act in that way. But I am at the early stages of programming initial programs with economic/learning strategies. So I don't know how promising it is. More details can be found spread out on my blog [3] [1] https://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.html https://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.htm... [2] https://github.com/eb4890/agorint/ https://github.com/eb4890/agorint/ [3] https://improvingautonomy.wordpress.com/2017/08/13/introduction-to-agorint-part-1 https://improvingautonomy.wordpress.com/2017/08/13/introduct...