11 ms·

A nice thought-parody of arguments about creating General AI. It would be nice to have an updated version because the option 'we just need more computing power

A nice thought-parody of arguments about creating General AI. It would be nice to have an updated version because the option 'we just need more computing power' (and training data) is now working pretty well for many machine learning domains (i.e. deeplearning).

adrusi 10y ago
Sure, throwing more computing power at a problem that's already conceptually solved, more or less, is going to yield results. But that won't help for creating general ai because it's not conceptually solved.
- arithma 10y ago
  It may still come out, even if it's not conceptually solved. We do a lot of things that are not conceptually solved.
  - stinkytaco 10y ago
    Can you give an example of something we do that's not conceptually solved? It seems to me, (from a notably uneducated in AI perspective), that when dealing something deterministic like computing, general AI either works or it doesn't. You can't "do" something that's not conceptually solved in this domain, because what you are really doing is something else, albeit something that is very much like the goal.
    - Analemma_ 10y ago
      Humor.
      - adrusi 10y ago
        You mean humans doing humor? Because humans also do intelligence. Computers don't do humor.
        Analemma_ 10y ago
        What I meant by my comment was, I understood stinkytaco's comment as a request for a specific task that humans do (so, nothing as nebulous as "intelligence") that hasn't been "conceptually solved" with something like a walk through a search tree. Even Go was "conceptually solved" decades ago- it's "only" iterating over a search tree; we were just waiting for the hardware and tree-pruning algorithms to catch up. Humor and other human tasks can be more precisely defined but still don't even have theoretical algorithms to solve them.
        adrusi 10y ago
        I didn't mean that humans can't do things that haven't been conceptually solved. I agree that humans can do humor and that humor isn't conceptually solved, so I'll assume that you mean to say that we can make computers do humor, even though it isn't conceptually solved. I disagree that we can make computers do humor. Kobeya linked to a paper that shows unsupervised joke generation, but it's a stretch to call this humor. I haven't explored beyond what's in that article, but the examples of jokes produced do little to convince me that this program is doing humor: I like my relationships like I like my source, open I like my coffee like I like my war, cold I like my boys like I like my sectors, bad Show me an AI that can produce original joke forms and I'll reevaluate. Go was not conceptually solved decades ago if it required the development of new tree-pruning algorithms to work. Improving an algorithm is conceptual innovation. If it could have worked with only a few orders of magnitude difference in performance, or with only marginally inferior results without new algorithms then maybe it was conceptually solved decades ago, else no.
        kobeya 10y ago
        You study these things on fixed forms because it is easier to analyze and understand what is going on. But the extension to general joke-making is straight forward, provided you have working NLP (a big if!): data-mine the concept graph for thoughts/memes which have similar structural content but surprisingly (in the technical sense) different meanings, then construct a sentence that maximizes probability during reading of the first interpretation but ends with the second. This captures basically all humor from puns to "I like my X like I like Y: Z" to most forms of stand-up comedy.
      - stinkytaco 10y ago
        Interesting. Of course, computers tell bad jokes, I'm told. Visual processing also comes to mind. Computers can "see", but it's very difficult to understand the many rules humans use to parse their environment, thus computers are not yet at human level of visual processing. Makes me wonder if those things aren't "solved" by our brains, but we just don't quite know how. Another thing worth noting is that I know people who have no sense of humor. They simply do not understand jokes or sarcasm.
      - kobeya 10y ago
        Unsupervised joke generation from big data: http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervised.pdf http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervise...
    - AndrewKemendo 10y ago
      Well we understand it (mostly) now, but at the time it was created: Specifics of aerodynamics required for flight.[1] The Wright brothers basically mimicked nature and tested till they got it right. They didn't use Bernoulli's equation to create a NACA airfoil and L/D max calculations. [2]. To that end, in one of my undergraduate Aeronautics classes, when we were doing a bunch of Bernoulli equations someone calculated that based on that approach helicopters couldn't actually fly. To which our instructor said, "Oh yea, helicopters fly by magic." [1]https://secretofflight.wordpress.com/incorrect-theories/ https://secretofflight.wordpress.com/incorrect-theories/ [2]https://wright.nasa.gov/overview.htm https://wright.nasa.gov/overview.htm
      - adrusi 10y ago
        This is a decent example, but it's not quite comparable to developing general AI. We can build helicopters without a full conceptual understanding of aerodynamics, but we can't produce a physical simulation which will accurately determine which helicopter designs will and will not fly for any given design without a full understanding of aerodynamics under the conditions in which helicopters operate. Creating aerodynamics simulation is creating an aerodynamic system. Creating a general intelligence is creating an intelligent system. A better analog to creating a helicopter using incomplete understanding of aerodynamics is something like creating a system that can discriminate between what is and isn't a general intelligence. We don't have such a system, but it at least doesn't seem like the sort of thing that would require a full understanding of intelligence.
        kobeya 10y ago
        That seems at best totally unrelated? You've asserted that general AI is some sort of different category than other things which we can and have built without understanding the fundamentals of their operation. But you haven't actually provided any real evidence of that.
- kobeya 10y ago
  You have a basis for that assertion? The human mind is basically lots of computing power, lots of data, and simple rules -- since all growth and learning is local, it must be simple in that sense at least. Yes we don't understand the neuroscience fully yet, but we do understand the ways in which the underlying mechanisms must be constrained. And we routinely build software programs with structure and rules that are vastly more complex than the neurons that make up our gray matter. Maybe automatic hyper-parameter optimization of adversarial networks is all that it takes for general intelligence. If you look up the conference proceedings of NIPS or AAAI you'll see major progress being made on basically every type of learning and memory you can find in a psychological textbook. Of course these are controlled experiments being done, in the technical sense, and not agglomerative. But what's stopping us from combining a bunch of these algorithms together, or better yet a base framework that generically supports them all, and making it self-reflective? Just computing power and available data, really. So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up, but the reality on the ground is otherwise.
  - justinpombrio 10y ago
    > So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up These people wouldn't happen to be domain experts...?
    - DubiousPusher 10y ago
      The question still seems up for argument but there is a fairly popular view that "General AI" whatever that is, is an aergent property many local system working in concert.
  - adrusi 10y ago
    If you think that we have all the conceptual understanding to assemble a general artificial intelligence then you should be able to give an outline of how it would work — beyond just lots of computing power and data, that doesn't differentiate the problem form the problem of object classification or natural language processing. Maybe what you mean to say is that once we have the computing power and data collection technology required, so that researches can experiment, the unsolved conceptual problems will become easy to solve. Do you have a basis for that assertion? Regardless, if that's what you meant then the fact remains that the conceptual problems are not all solved. I won't say that it's inconceivable that a sufficiently large and powerful neural net, with a simple structure based on simple rules, given a very comprehensive set of training data, could acheive a level of general intelligence. But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure resembling a neural net with simple rules for individual node (maybe?). It's the result of evolutionary processes, which means that we should expect it to have extremely hard-to-comprehend properties (example [0]). We've seen evolutionary processes produce intelligence, but not learning processes. Again, that's not to say that evolutionary processes are necessary, just that we can be sure that they can lead to intelligence, and that we can't be sure for other methods. [0] https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/
- btilly 10y ago
  A surprising number of the things that have been solved in the past decade with more computing powers were things that were not conceptually solved, and which we had no good ideas how to solve. A good example is Google Translate. We stopped trying to "understand" language and just threw a large dataset at pattern matching. And it worked amazingly well. For other things we throw ideas of "this might improve what we can already do" and it winds up effectively solved, whether or not we understand it. This happened, for example, with Alpha-Go. So "conceptually solved" and "AI can do it" are pretty much disconnected.
  - AndrewKemendo 10y ago
    This was actually my breakthrough for Machine Learning. Prior to really understanding ML and DL I thought it was just an update to expert systems and that we still needed to "understand intelligence" before we created it. The genius of the RL feedback loop makes me think we can get to AGI without understanding human intelligence holistically.
  - bsaul 10y ago
    I still don't understand why google translate is often cited as something that works. I've never seen it perform much better that just looking every word in a dictionary (which i always end up doing, since you're never sure it picked the correct translation for the specific context) To me google translate seems like the perfect illustration of the shortcomings of this purely data driven approach... Alpha go, on the other hand is a much much more elaborated algorithm ( from my very limited knowledge) and it does produce amazing results.
    - btilly 10y ago
      The reason why you don't understand is because it's shortcomings are your baseline, while more experienced people's baseline was people spending tens of millions to get something that only worked for toy examples. Yes, I can list plenty of cases where translation fails. For example not understanding that in a conversation about specialist topic X (eg Ruby programming, or the game of G) that seemingly common terms have specific meanings. But I can also use it on a random newspaper article in a different language and expect to understand the result. This was something that I did not expect to happen in my lifetime.
- Houshalter 10y ago
  I disagree strongly with this view that hard problems need to be "conceptually solved" in a vacuum first. That experimentation and resources don't matter. Computing increases have enabled a ton more experimentation with AI. Experiments that would have taken years to do, can be done in hours now. Experimentation lets scientists develop intuition about their algorithms. And get a sense of what will and won't work. And have a better mental model of the problem. Not to mention the benefit of having 10x or more funding and researchers working on it now that results have been shown. It's a common myth that there has been little or no innovation in NNs and that it's just a matter of computing power. Taking the best algorithms of the 90s and running them on modern hardware would still give you poor results. For instance, most of the 90s research was on very shallow nets, they didn't know how to properly train deep ones.
  - obastani 10y ago
    What innovations do you have in mind? LSTMs and convnets were both invented in the 90s. Stochastic gradient descent, momentum, etc. have been around for decades. I feel like it's quite the opposite of what you are saying -- in the 2000s there was a lot of innovation, but it mostly turned out to be unnecessary. What has improved is the variety of architectures, but I don't think there has been anything fundamentally new after LSTMs. The only exception I can think of is generative adversarial networks, and even ideas behind adversarial training have been around in other domains for many years now.
    - deleted 10y ago
      [deleted]
- VladKovac 10y ago
  I believe the author himself said that computation improvements does make it easier to solve general AI. I think in value alignment theory there are still problems that they don't know how to do properly even with infinite computation though.

adrusi 10y ago

Sure, throwing more computing power at a problem that's already conceptually solved, more or less, is going to yield results. But that won't help for creating general ai because it's not conceptually solved.

arithma 10y ago
It may still come out, even if it's not conceptually solved. We do a lot of things that are not conceptually solved.
- stinkytaco 10y ago
  Can you give an example of something we do that's not conceptually solved? It seems to me, (from a notably uneducated in AI perspective), that when dealing something deterministic like computing, general AI either works or it doesn't. You can't "do" something that's not conceptually solved in this domain, because what you are really doing is something else, albeit something that is very much like the goal.
  - Analemma_ 10y ago
    Humor.
    - adrusi 10y ago
      You mean humans doing humor? Because humans also do intelligence. Computers don't do humor.
      - Analemma_ 10y ago
        What I meant by my comment was, I understood stinkytaco's comment as a request for a specific task that humans do (so, nothing as nebulous as "intelligence") that hasn't been "conceptually solved" with something like a walk through a search tree. Even Go was "conceptually solved" decades ago- it's "only" iterating over a search tree; we were just waiting for the hardware and tree-pruning algorithms to catch up. Humor and other human tasks can be more precisely defined but still don't even have theoretical algorithms to solve them.
        adrusi 10y ago
        I didn't mean that humans can't do things that haven't been conceptually solved. I agree that humans can do humor and that humor isn't conceptually solved, so I'll assume that you mean to say that we can make computers do humor, even though it isn't conceptually solved. I disagree that we can make computers do humor. Kobeya linked to a paper that shows unsupervised joke generation, but it's a stretch to call this humor. I haven't explored beyond what's in that article, but the examples of jokes produced do little to convince me that this program is doing humor: I like my relationships like I like my source, open I like my coffee like I like my war, cold I like my boys like I like my sectors, bad Show me an AI that can produce original joke forms and I'll reevaluate. Go was not conceptually solved decades ago if it required the development of new tree-pruning algorithms to work. Improving an algorithm is conceptual innovation. If it could have worked with only a few orders of magnitude difference in performance, or with only marginally inferior results without new algorithms then maybe it was conceptually solved decades ago, else no.
        kobeya 10y ago
        You study these things on fixed forms because it is easier to analyze and understand what is going on. But the extension to general joke-making is straight forward, provided you have working NLP (a big if!): data-mine the concept graph for thoughts/memes which have similar structural content but surprisingly (in the technical sense) different meanings, then construct a sentence that maximizes probability during reading of the first interpretation but ends with the second. This captures basically all humor from puns to "I like my X like I like Y: Z" to most forms of stand-up comedy.
    - stinkytaco 10y ago
      Interesting. Of course, computers tell bad jokes, I'm told. Visual processing also comes to mind. Computers can "see", but it's very difficult to understand the many rules humans use to parse their environment, thus computers are not yet at human level of visual processing. Makes me wonder if those things aren't "solved" by our brains, but we just don't quite know how. Another thing worth noting is that I know people who have no sense of humor. They simply do not understand jokes or sarcasm.
    - kobeya 10y ago
      Unsupervised joke generation from big data: http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervised.pdf http://homepages.inf.ed.ac.uk/s0894589/petrovic13unsupervise...
  - AndrewKemendo 10y ago
    Well we understand it (mostly) now, but at the time it was created: Specifics of aerodynamics required for flight.[1] The Wright brothers basically mimicked nature and tested till they got it right. They didn't use Bernoulli's equation to create a NACA airfoil and L/D max calculations. [2]. To that end, in one of my undergraduate Aeronautics classes, when we were doing a bunch of Bernoulli equations someone calculated that based on that approach helicopters couldn't actually fly. To which our instructor said, "Oh yea, helicopters fly by magic." [1]https://secretofflight.wordpress.com/incorrect-theories/ https://secretofflight.wordpress.com/incorrect-theories/ [2]https://wright.nasa.gov/overview.htm https://wright.nasa.gov/overview.htm
    - adrusi 10y ago
      This is a decent example, but it's not quite comparable to developing general AI. We can build helicopters without a full conceptual understanding of aerodynamics, but we can't produce a physical simulation which will accurately determine which helicopter designs will and will not fly for any given design without a full understanding of aerodynamics under the conditions in which helicopters operate. Creating aerodynamics simulation is creating an aerodynamic system. Creating a general intelligence is creating an intelligent system. A better analog to creating a helicopter using incomplete understanding of aerodynamics is something like creating a system that can discriminate between what is and isn't a general intelligence. We don't have such a system, but it at least doesn't seem like the sort of thing that would require a full understanding of intelligence.
      - kobeya 10y ago
        That seems at best totally unrelated? You've asserted that general AI is some sort of different category than other things which we can and have built without understanding the fundamentals of their operation. But you haven't actually provided any real evidence of that.
kobeya 10y ago
You have a basis for that assertion? The human mind is basically lots of computing power, lots of data, and simple rules -- since all growth and learning is local, it must be simple in that sense at least. Yes we don't understand the neuroscience fully yet, but we do understand the ways in which the underlying mechanisms must be constrained. And we routinely build software programs with structure and rules that are vastly more complex than the neurons that make up our gray matter. Maybe automatic hyper-parameter optimization of adversarial networks is all that it takes for general intelligence. If you look up the conference proceedings of NIPS or AAAI you'll see major progress being made on basically every type of learning and memory you can find in a psychological textbook. Of course these are controlled experiments being done, in the technical sense, and not agglomerative. But what's stopping us from combining a bunch of these algorithms together, or better yet a base framework that generically supports them all, and making it self-reflective? Just computing power and available data, really. So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up, but the reality on the ground is otherwise.
- justinpombrio 10y ago
  > So no, it's not at all clear that this is a problem that is "not conceptually solved." People keep saying that whenever the subject comes up These people wouldn't happen to be domain experts...?
  - DubiousPusher 10y ago
    The question still seems up for argument but there is a fairly popular view that "General AI" whatever that is, is an aergent property many local system working in concert.
- adrusi 10y ago
  If you think that we have all the conceptual understanding to assemble a general artificial intelligence then you should be able to give an outline of how it would work — beyond just lots of computing power and data, that doesn't differentiate the problem form the problem of object classification or natural language processing. Maybe what you mean to say is that once we have the computing power and data collection technology required, so that researches can experiment, the unsolved conceptual problems will become easy to solve. Do you have a basis for that assertion? Regardless, if that's what you meant then the fact remains that the conceptual problems are not all solved. I won't say that it's inconceivable that a sufficiently large and powerful neural net, with a simple structure based on simple rules, given a very comprehensive set of training data, could acheive a level of general intelligence. But we don't have evidence of intelligence arising that way in the past. What we have is the human brain, an incredibly complex structure resembling a neural net with simple rules for individual node (maybe?). It's the result of evolutionary processes, which means that we should expect it to have extremely hard-to-comprehend properties (example [0]). We've seen evolutionary processes produce intelligence, but not learning processes. Again, that's not to say that evolutionary processes are necessary, just that we can be sure that they can lead to intelligence, and that we can't be sure for other methods. [0] https://www.damninteresting.com/on-the-origin-of-circuits/ https://www.damninteresting.com/on-the-origin-of-circuits/
btilly 10y ago
A surprising number of the things that have been solved in the past decade with more computing powers were things that were not conceptually solved, and which we had no good ideas how to solve. A good example is Google Translate. We stopped trying to "understand" language and just threw a large dataset at pattern matching. And it worked amazingly well. For other things we throw ideas of "this might improve what we can already do" and it winds up effectively solved, whether or not we understand it. This happened, for example, with Alpha-Go. So "conceptually solved" and "AI can do it" are pretty much disconnected.
- AndrewKemendo 10y ago
  This was actually my breakthrough for Machine Learning. Prior to really understanding ML and DL I thought it was just an update to expert systems and that we still needed to "understand intelligence" before we created it. The genius of the RL feedback loop makes me think we can get to AGI without understanding human intelligence holistically.
- bsaul 10y ago
  I still don't understand why google translate is often cited as something that works. I've never seen it perform much better that just looking every word in a dictionary (which i always end up doing, since you're never sure it picked the correct translation for the specific context) To me google translate seems like the perfect illustration of the shortcomings of this purely data driven approach... Alpha go, on the other hand is a much much more elaborated algorithm ( from my very limited knowledge) and it does produce amazing results.
  - btilly 10y ago
    The reason why you don't understand is because it's shortcomings are your baseline, while more experienced people's baseline was people spending tens of millions to get something that only worked for toy examples. Yes, I can list plenty of cases where translation fails. For example not understanding that in a conversation about specialist topic X (eg Ruby programming, or the game of G) that seemingly common terms have specific meanings. But I can also use it on a random newspaper article in a different language and expect to understand the result. This was something that I did not expect to happen in my lifetime.
Houshalter 10y ago
I disagree strongly with this view that hard problems need to be "conceptually solved" in a vacuum first. That experimentation and resources don't matter. Computing increases have enabled a ton more experimentation with AI. Experiments that would have taken years to do, can be done in hours now. Experimentation lets scientists develop intuition about their algorithms. And get a sense of what will and won't work. And have a better mental model of the problem. Not to mention the benefit of having 10x or more funding and researchers working on it now that results have been shown. It's a common myth that there has been little or no innovation in NNs and that it's just a matter of computing power. Taking the best algorithms of the 90s and running them on modern hardware would still give you poor results. For instance, most of the 90s research was on very shallow nets, they didn't know how to properly train deep ones.
- obastani 10y ago
  What innovations do you have in mind? LSTMs and convnets were both invented in the 90s. Stochastic gradient descent, momentum, etc. have been around for decades. I feel like it's quite the opposite of what you are saying -- in the 2000s there was a lot of innovation, but it mostly turned out to be unnecessary. What has improved is the variety of architectures, but I don't think there has been anything fundamentally new after LSTMs. The only exception I can think of is generative adversarial networks, and even ideas behind adversarial training have been around in other domains for many years now.
  - deleted 10y ago
    [deleted]
VladKovac 10y ago
I believe the author himself said that computation improvements does make it easier to solve general AI. I think in value alignment theory there are still problems that they don't know how to do properly even with infinite computation though.