4 ms·

I am not trained in any type of field that is remotely related to AI research or engineering. Outside of some basic ML projects at work, I am not well versed i

I am not trained in any type of field that is remotely related to AI research or engineering. Outside of some basic ML projects at work, I am not well versed in the practical application of AI technologies. But I do wonder a few things about AI: - Have we made real, technological progress over the last 50 years or are we just leveraging far greater computing power and the ability to collect much larger data sets to run statistical analysis on? - Will general purpose AI consist of essentially layers and layers of AI's that can handle progressively more abstract inputs, models, and patterns? For instance, the lowest level AI is what we see today - a very powerful tool but bound to a specific use case. One layer up may be able to combine inputs from a dozen 1st tier AI's to generalize a tiny bit more on the individual use cases and can deal with a tiny bit more ambiguity. One level up will evaluate inputs from a dozen level 2 AI's and so on. With the final top layer (perhaps millions of levels up) resembling general purpose processing similar to a human brain. What if this model ended up producing true general purpose AI, but the amount of input synthesizing and modeling required so much processing power that the speed at which general purpose AI could operate is no faster than a human brain? - Can we achieve general purpose AI through purely algorithmic means, or will we need to implement a hybrid biological model to achieve real breakthroughs? If we could accomplish this, would we understand the detailed mechanisms of the biological component of the hybrid or would it forever remain a black box that we just tap into? Anyway, not sure this adds a whole lot to the specific discussion on how best to measure AI progress, but they're questions I've been pondering lately.

pnloyd 8y ago
Ya that's kind of an interesting question. With Moore's law starting to approach it's physical limitations it would seem AGI wouldn't be feasible with current algorithms. Not to mentions as your describing.. those "millions" of layers of narrow AI's sounds like impossible amount of work to do.. I don't think very many machine learning experts really believe that those techniques will lead to AGI.
- taeric 8y ago
  I'm curious just how true it is that Moore's law is starting to approach physical limitations. I just recently listened to some of Feynman's speeches collected in [1]. One of them was about how to place all of the works of an encyclopedia onto the tip of a pin. Did it cover anything we couldn't do today? No. But that was the point. Just using a somewhat naive view of the physical matter that you would be putting something on, it was possible to go quite dense. Imagine if we started going even denser. Do I suspect we are approaching limits? Certainly. Question is more of just how much further we can go. And will we need a dramatic shift of any sort before we could realize some extra distance? [1] https://www.audible.com/pd/Science-Technology/The-Pleasure-of-Finding-Things-Out-Audiobook/B00BSU83HI https://www.audible.com/pd/Science-Technology/The-Pleasure-o...
  - ghaff 8y ago
    >I'm curious just how true it is that Moore's law is starting to approach physical limitations. Well, Moore's "Law" in the narrow sense has been, to a large degree, about CMOS process scaling and that's clearly running into physical limits. There are other levers to get better economical performance--some of which come at the cost of extra work in software. For certain workloads, GPUs and TPUs have been an important work around. There almost certainly are further optimizations involving stacking and interconnects. Probably other application-tailored designs (which then have to have software tailored for them individually). But CMOS scaling has been such a powerful lever that there's legitimate concern that it may not be possible to replicate that kind of advance using other techniques.
    - taeric 8y ago
      Fair. In the original context, it is clear they were on CMOS, and yes we do seem to be nearing these limits quite quickly. I'm curious if/when we could/should move off of current CMOS techniques.
Erlich_Bachman 8y ago
> Will general purpose AI consist of essentially layers and layers of AI's that can handle progressively more abstract inputs, models, and patterns? This will very likely be the case. It has already sort of been the case in the AI that we have already built. The best understanding we have about the real brain - is something similar to that case. Notably, we also want it to be the case, we will try to make it work first in that way, before any other way, because that's the only way for us to be able to understand it. We would rather have a machine that we can still understand at least on some levels, instead of one which is just a black box. So of course we will want to split it up into several levels which will be easier to understand, at least individually. > Can we achieve general purpose AI through purely algorithmic means No one knows. The only way to know is to actually build something, then progress slowly by iterating and doing more experimentation, trying to implement new ideas etc. Focus the attention on the most lucrative ideas. This is what the AI community is doing right now. > What if this model ended up producing true general purpose AI, but the amount of input synthesizing and modeling required so much processing power that the speed at which general purpose AI could operate is no faster than a human brain? I don't think it is even a common idea that it is a specific goal of general AI research to make it "faster" than a human brain. Faster for what? Thinking faster? Being smarter - sure? Being more agile in terms of not requiring sleep, or being ever-vigilant while things like driving for exapmle, sure. But faster? What would that even mean in practice? We already have machines that are faster than humans at physical tasks, they don't require GAI. Producing a brain which works at the same speed as a human would still be a tremendous achievement and would transform the life on earth as we know it. Even if it would be 10 times slower than a human, it would be an achievement. Even if this is a slow AI, we could just copy it multiple times and make them work together. Like people are working together now. If you only have 10 people in the world sufficiently knowledgable in a field, they can only do so much work and progress. If you could copy each one of them 1000 times, those copies could pursue different paths simultaneously and the progress would be much faster. Or why not copy a perfect store clerk so that no people would need to do that anymore? Ability to backup and restore certain snapshots of a brain would be amazing. And of course, if there is a physical way to speed up a brain - it would certainly be much easier once you have a working brain, than right now would proper understanding of how it even works.
- 8y ago

pnloyd 8y ago

Ya that's kind of an interesting question. With Moore's law starting to approach it's physical limitations it would seem AGI wouldn't be feasible with current algorithms. Not to mentions as your describing.. those "millions" of layers of narrow AI's sounds like impossible amount of work to do.. I don't think very many machine learning experts really believe that those techniques will lead to AGI.

taeric 8y ago
I'm curious just how true it is that Moore's law is starting to approach physical limitations. I just recently listened to some of Feynman's speeches collected in [1]. One of them was about how to place all of the works of an encyclopedia onto the tip of a pin. Did it cover anything we couldn't do today? No. But that was the point. Just using a somewhat naive view of the physical matter that you would be putting something on, it was possible to go quite dense. Imagine if we started going even denser. Do I suspect we are approaching limits? Certainly. Question is more of just how much further we can go. And will we need a dramatic shift of any sort before we could realize some extra distance? [1] https://www.audible.com/pd/Science-Technology/The-Pleasure-of-Finding-Things-Out-Audiobook/B00BSU83HI https://www.audible.com/pd/Science-Technology/The-Pleasure-o...
- ghaff 8y ago
  >I'm curious just how true it is that Moore's law is starting to approach physical limitations. Well, Moore's "Law" in the narrow sense has been, to a large degree, about CMOS process scaling and that's clearly running into physical limits. There are other levers to get better economical performance--some of which come at the cost of extra work in software. For certain workloads, GPUs and TPUs have been an important work around. There almost certainly are further optimizations involving stacking and interconnects. Probably other application-tailored designs (which then have to have software tailored for them individually). But CMOS scaling has been such a powerful lever that there's legitimate concern that it may not be possible to replicate that kind of advance using other techniques.
  - taeric 8y ago
    Fair. In the original context, it is clear they were on CMOS, and yes we do seem to be nearing these limits quite quickly. I'm curious if/when we could/should move off of current CMOS techniques.

Erlich_Bachman 8y ago

> Will general purpose AI consist of essentially layers and layers of AI's that can handle progressively more abstract inputs, models, and patterns? This will very likely be the case. It has already sort of been the case in the AI that we have already built. The best understanding we have about the real brain - is something similar to that case. Notably, we also want it to be the case, we will try to make it work first in that way, before any other way, because that's the only way for us to be able to understand it. We would rather have a machine that we can still understand at least on some levels, instead of one which is just a black box. So of course we will want to split it up into several levels which will be easier to understand, at least individually. > Can we achieve general purpose AI through purely algorithmic means No one knows. The only way to know is to actually build something, then progress slowly by iterating and doing more experimentation, trying to implement new ideas etc. Focus the attention on the most lucrative ideas. This is what the AI community is doing right now. > What if this model ended up producing true general purpose AI, but the amount of input synthesizing and modeling required so much processing power that the speed at which general purpose AI could operate is no faster than a human brain? I don't think it is even a common idea that it is a specific goal of general AI research to make it "faster" than a human brain. Faster for what? Thinking faster? Being smarter - sure? Being more agile in terms of not requiring sleep, or being ever-vigilant while things like driving for exapmle, sure. But faster? What would that even mean in practice? We already have machines that are faster than humans at physical tasks, they don't require GAI. Producing a brain which works at the same speed as a human would still be a tremendous achievement and would transform the life on earth as we know it. Even if it would be 10 times slower than a human, it would be an achievement. Even if this is a slow AI, we could just copy it multiple times and make them work together. Like people are working together now. If you only have 10 people in the world sufficiently knowledgable in a field, they can only do so much work and progress. If you could copy each one of them 1000 times, those copies could pursue different paths simultaneously and the progress would be much faster. Or why not copy a perfect store clerk so that no people would need to do that anymore? Ability to backup and restore certain snapshots of a brain would be amazing. And of course, if there is a physical way to speed up a brain - it would certainly be much easier once you have a working brain, than right now would proper understanding of how it even works.

8y ago