15 ms·

Something tells me people that write these articles have never read a paper from e.g. NIPS or pick any top tier conference on cutting edge research, heck I woul

Something tells me people that write these articles have never read a paper from e.g. NIPS or pick any top tier conference on cutting edge research, heck I would go so far as to say they don’t even know how to write an image classifier for MNIST using Keras if their life depended on it. Universal function approximators are not about to take over the world.

garmaine 8y ago
Devil’s advocate: success is a step function. AI isn’t going to take over the world until it is able to, and then it very rapidly does.
- goatlover 8y ago
  Devil's devil's advocate. The world is too complicated to be reduced to a function. AI isn't going to take over the world anymore than anyone else has (governments and organizations included).
  - gowld 8y ago
    "the world" is vague. Governments certainly have taken over various worlds.
  - canjobear 8y ago
    > The world is too complicated to be reduced to a function. I can only interpret this as a mystical claim...
    - cornholio 8y ago
      The world is too complicated to be reduced to a function that does not exhibit general intelligence. At minimum, able to formulate rational strategies in arbitrary games against other rational adversaries.
    - goatlover 8y ago
      The mystical claim is that the world is like a board game where we can extrapolate from the success of AlphaGo/Zero to AGI taking over the world. That's just pure speculation because nobody has a clue what such an intelligence would look like. I can imagine using the mind infinity stone to take over the world, but that doesn't make it realistic.
      - red75prime 8y ago
        We have a working model of general intelligence. It runs on 20 watts, takes 20 years to became somewhat usable, and is known to cause local disasters.
  - fossuser 8y ago
    Take the human general intelligence learning architecture and scale it up to a billion operations a second. The trick with AGI is that it can learn faster, and humans (even with our limitations) did take over the world.
    - goatlover 8y ago
      So one AGI is like a billion human beings, so a little bit less than China or India. And like countries, there will be other AGIs. An AGI isn't born in a vacuum, it comes into existence in a world advanced enough for such a tech to be developed. As such, it's reasonable to suppose there will limitations for AGIs when it decides that world conquest is a worthy goal.
      - fossuser 8y ago
        It's not like one billion human beings. Humans are limited by a lot of things like motivation and communication which is likely distinct from the general problem solving architecture itself. Humans also have goals and an evolved morality that guides their thinking. Humans share a similar type of intelligence and understanding of goals, culture, right and wrong. You don't get this for free with AGI. The point of the human example is that there already exists a general problem solving architecture you can scale up (it's not impossible and there are brains everywhere in nature). If you increase operations per second many magnitudes you can have all human learning over thousands of years compressed into a couple of hours. You see this loosely in narrow spaces now with things like Alpha Go Zero. There may not be other AGIs if there's an intelligence explosion and the first one improves itself very quickly. If its utility function is not aligned with human goals then it could end up turning all matter into paperclips - not because it's evil, but because it's hard to set goals tied to human morality (when humans don't even agree in every case) and it happens to be configured such that making paperclips maxes out its reward function by accident. It may be possible to make the unsafe version before making the safe one, since it seems harder to make an AGI that's aligned with human goals than just an AGI in general.
  - garmaine 8y ago
    Organizations of humans (governments) seem to have conquered the world pretty well, outside of a scant few lawless areas. Even then there's usually some local warlord maintaining order, just not a nation state. I'm not sure what you're trying to say?
- laichzeit0 8y ago
  What seems more likely at this point is that: machine translation will get so good that it will be better at translating between languages than any human can; image classifiers will become so good that they will be better at classifying images than any human can; self-driving cars will become so good that they will be better at driving than any human can; etc. etc. I think the mismatch comes when people looking from the outside who have no real understanding of how the internals work see this progress in a very narrow domain and extrapolate that idea into thinking that something more general is happening. When you understand the trick, it stops being magic.
  - klipt 8y ago
    If a machine can drive a car, how much harder is it to drive a tank? If a machine can generate strategies to win simulated wars, how much harder is it to generate strategies to win real wars? If machines can out compete people on 50% of jobs, how much harder is it to out compete people on 100% of jobs?
    - YeGoblynQueenne 8y ago
      Well, even for humans driving a car doesn't mean you can drive a tank; being good at Rome: Total War or Starcraft doesn't mean you can win a real war; and we're a long way away from "machines" being able to outcompete 50% of humans at their jobs.
  - red75prime 8y ago
    Some day we will understand how we do our tricks though.
    - garmaine 8y ago
      Or alternatively, how to accomplish the same thing in an easier way -- we don't fly in commercial aircraft by flapping feathered wings, after all. And then magic will be dissolved and we might find that general AI isn't really all that much harder than narrow AI.
  - 8y ago
- YeGoblynQueenne 8y ago
  >> AI isn’t going to take over the world until it is able to, and then it very rapidly does. The question is, of course- how? How will AI become able to take over the world? This is an important question to ask because predictions of AI rapidly becoming capable of taking over the world tend to flare up in times of progress in AI performance. For example, the above article is basically based on the preimse that the latest developments, i.e. deep learning, are sufficient to give some nation-state technological supremacy by leading directly to AGI. However, there is no clear path from deep learning, however powerful it may be as a set of techniques to train classifiers, to AGI, which (well, probably) entails much more than powerful classification. For instance, deep learning can't really do inference. It seems reasonable to assume that AGI will need the ability to reason about the objects it can recognise and the relations between them. Deep learning can't do causal analysis- like Judea Pearl has been screaming from the rooftops lately. It seems reasonable again to assume that AGI will need to model causal relations. Deep learning can't do semantics. AGI will need to have some handle on semantics. And so on and so forth. In general, anyone that claims that X current development in AI risks inadvertently producing AGI, must explain how this is supposed to happen- not just wave their hands about and claim that, if you throw enough compute and enough data at the problem, magic will happen and AGI will turn us all into paperclips.
red75prime 8y ago
What about reinforcement learned universal function approximators?
- visarga 8y ago
  You can hardly train RL in the real world (as opposed to simulation) because of its high sample complexity.
  - red75prime 8y ago
    Exacerbated by sparse rewards. Well, it is being tackled as everything else. [0] for example. [0] https://arxiv.org/pdf/1802.10567.pdf https://arxiv.org/pdf/1802.10567.pdf
canjobear 8y ago
I'm always a bit confused when people call neural networks "universal function approximators" as if that makes them trivial or weak. Suppose we have a really strong universal function approximator---stronger than current neural networks, whose generalization properties are not really that great in the grand scheme of things as of 2018. Tell it to approximate the action policy that maximizes some overly-simplistic geopolitical objective function, like GDP or territory controlled at time t+1. It doesn't seem at all obvious that this thing could not take over the world or at least cause significant havoc if given sufficient resources.
- schoen 8y ago
  At least, a lot of things that have conventionally been understood as very hard for AI (like catching a ball or playing Go) can be viewed as function approximations!
- YeGoblynQueenne 8y ago
  But what are "sufficient resources" in this case? How do you train a universal function approximator, even a good one, in, say "increasing GDP"? Where do you get the examples for that? The problem is that with such a vague objective as "maximise GDP" or "maximise controlled territory" you need to train a model that is extremely broad in scope - because the objective might be narrow, but the steps to realise it are extremely varied. In practice, you're trying to approximate a function that is outputting the state of the entire world at each time step. Good luck with achieving that in practice. Edit: The bit about collecting examples is not a trivial problem. Note that the successes of deep learning so far are in domains where not only the objective is well defined ("choose one of n categorical labels") but also the data associated with the objectives is easy to collect and has an obvious relation to the outputs. Say, if you want to train an image classifier to recognise images of dogs- obviously you need to collect images of dogs. This is not the case in "increase GDP" type objectives, where it is not even clear what exactly influences a country's GDP. In principle, you could feed the entire world as examples to the model, but in practice, that's just unfeasible.
  - ShabbosGoy 8y ago
AndrewKemendo 8y ago
What's especially shocking about your statement is the complete lack of research you did on Ian before posting this. https://www.ianhogarth.com/about/ https://www.ianhogarth.com/about/ https://en.m.wikipedia.org/wiki/Ian_Hogarth https://en.m.wikipedia.org/wiki/Ian_Hogarth Ignoring the unsubstantiated ad hominem for a minute though, it's getting frustrating that the mere introduction of the topic of discussing possible AI futures causes immediate derision. It's as though the corporate research community, for which I'm associated, is vehemently against even discussing AGI. Whereas Barto, Bengio, Hassabis etc... are happy to discuss it in reasonable ways. The author seems to be discussing it reasonably and not making crazy kurzweilesque prognostication. Everyone at the Dartmouth Workshop wanted to create human level intelligence. Let's stop pretending that's not still the goal. OpenAI and DeepMind have that as explicit goals and if you dig into any serious AI researchers they say that's the goal vector. So where's the beef? I think it comes from a combination of being scared of overhyping AI and causing the next AI winter, and some form of gatekeeping.

garmaine 8y ago

Devil’s advocate: success is a step function. AI isn’t going to take over the world until it is able to, and then it very rapidly does.

goatlover 8y ago
Devil's devil's advocate. The world is too complicated to be reduced to a function. AI isn't going to take over the world anymore than anyone else has (governments and organizations included).
- gowld 8y ago
  "the world" is vague. Governments certainly have taken over various worlds.
- canjobear 8y ago
  > The world is too complicated to be reduced to a function. I can only interpret this as a mystical claim...
  - cornholio 8y ago
    The world is too complicated to be reduced to a function that does not exhibit general intelligence. At minimum, able to formulate rational strategies in arbitrary games against other rational adversaries.
  - goatlover 8y ago
    The mystical claim is that the world is like a board game where we can extrapolate from the success of AlphaGo/Zero to AGI taking over the world. That's just pure speculation because nobody has a clue what such an intelligence would look like. I can imagine using the mind infinity stone to take over the world, but that doesn't make it realistic.
    - red75prime 8y ago
      We have a working model of general intelligence. It runs on 20 watts, takes 20 years to became somewhat usable, and is known to cause local disasters.
- fossuser 8y ago
  Take the human general intelligence learning architecture and scale it up to a billion operations a second. The trick with AGI is that it can learn faster, and humans (even with our limitations) did take over the world.
  - goatlover 8y ago
    So one AGI is like a billion human beings, so a little bit less than China or India. And like countries, there will be other AGIs. An AGI isn't born in a vacuum, it comes into existence in a world advanced enough for such a tech to be developed. As such, it's reasonable to suppose there will limitations for AGIs when it decides that world conquest is a worthy goal.
    - fossuser 8y ago
      It's not like one billion human beings. Humans are limited by a lot of things like motivation and communication which is likely distinct from the general problem solving architecture itself. Humans also have goals and an evolved morality that guides their thinking. Humans share a similar type of intelligence and understanding of goals, culture, right and wrong. You don't get this for free with AGI. The point of the human example is that there already exists a general problem solving architecture you can scale up (it's not impossible and there are brains everywhere in nature). If you increase operations per second many magnitudes you can have all human learning over thousands of years compressed into a couple of hours. You see this loosely in narrow spaces now with things like Alpha Go Zero. There may not be other AGIs if there's an intelligence explosion and the first one improves itself very quickly. If its utility function is not aligned with human goals then it could end up turning all matter into paperclips - not because it's evil, but because it's hard to set goals tied to human morality (when humans don't even agree in every case) and it happens to be configured such that making paperclips maxes out its reward function by accident. It may be possible to make the unsafe version before making the safe one, since it seems harder to make an AGI that's aligned with human goals than just an AGI in general.
- garmaine 8y ago
  Organizations of humans (governments) seem to have conquered the world pretty well, outside of a scant few lawless areas. Even then there's usually some local warlord maintaining order, just not a nation state. I'm not sure what you're trying to say?
laichzeit0 8y ago
What seems more likely at this point is that: machine translation will get so good that it will be better at translating between languages than any human can; image classifiers will become so good that they will be better at classifying images than any human can; self-driving cars will become so good that they will be better at driving than any human can; etc. etc. I think the mismatch comes when people looking from the outside who have no real understanding of how the internals work see this progress in a very narrow domain and extrapolate that idea into thinking that something more general is happening. When you understand the trick, it stops being magic.
- klipt 8y ago
  If a machine can drive a car, how much harder is it to drive a tank? If a machine can generate strategies to win simulated wars, how much harder is it to generate strategies to win real wars? If machines can out compete people on 50% of jobs, how much harder is it to out compete people on 100% of jobs?
  - YeGoblynQueenne 8y ago
    Well, even for humans driving a car doesn't mean you can drive a tank; being good at Rome: Total War or Starcraft doesn't mean you can win a real war; and we're a long way away from "machines" being able to outcompete 50% of humans at their jobs.
- red75prime 8y ago
  Some day we will understand how we do our tricks though.
  - garmaine 8y ago
    Or alternatively, how to accomplish the same thing in an easier way -- we don't fly in commercial aircraft by flapping feathered wings, after all. And then magic will be dissolved and we might find that general AI isn't really all that much harder than narrow AI.
- 8y ago
YeGoblynQueenne 8y ago
>> AI isn’t going to take over the world until it is able to, and then it very rapidly does. The question is, of course- how? How will AI become able to take over the world? This is an important question to ask because predictions of AI rapidly becoming capable of taking over the world tend to flare up in times of progress in AI performance. For example, the above article is basically based on the preimse that the latest developments, i.e. deep learning, are sufficient to give some nation-state technological supremacy by leading directly to AGI. However, there is no clear path from deep learning, however powerful it may be as a set of techniques to train classifiers, to AGI, which (well, probably) entails much more than powerful classification. For instance, deep learning can't really do inference. It seems reasonable to assume that AGI will need the ability to reason about the objects it can recognise and the relations between them. Deep learning can't do causal analysis- like Judea Pearl has been screaming from the rooftops lately. It seems reasonable again to assume that AGI will need to model causal relations. Deep learning can't do semantics. AGI will need to have some handle on semantics. And so on and so forth. In general, anyone that claims that X current development in AI risks inadvertently producing AGI, must explain how this is supposed to happen- not just wave their hands about and claim that, if you throw enough compute and enough data at the problem, magic will happen and AGI will turn us all into paperclips.

red75prime 8y ago

What about reinforcement learned universal function approximators?

visarga 8y ago
You can hardly train RL in the real world (as opposed to simulation) because of its high sample complexity.
- red75prime 8y ago
  Exacerbated by sparse rewards. Well, it is being tackled as everything else. [0] for example. [0] https://arxiv.org/pdf/1802.10567.pdf https://arxiv.org/pdf/1802.10567.pdf

canjobear 8y ago

I'm always a bit confused when people call neural networks "universal function approximators" as if that makes them trivial or weak. Suppose we have a really strong universal function approximator---stronger than current neural networks, whose generalization properties are not really that great in the grand scheme of things as of 2018. Tell it to approximate the action policy that maximizes some overly-simplistic geopolitical objective function, like GDP or territory controlled at time t+1. It doesn't seem at all obvious that this thing could not take over the world or at least cause significant havoc if given sufficient resources.

schoen 8y ago
At least, a lot of things that have conventionally been understood as very hard for AI (like catching a ball or playing Go) can be viewed as function approximations!
YeGoblynQueenne 8y ago
But what are "sufficient resources" in this case? How do you train a universal function approximator, even a good one, in, say "increasing GDP"? Where do you get the examples for that? The problem is that with such a vague objective as "maximise GDP" or "maximise controlled territory" you need to train a model that is extremely broad in scope - because the objective might be narrow, but the steps to realise it are extremely varied. In practice, you're trying to approximate a function that is outputting the state of the entire world at each time step. Good luck with achieving that in practice. Edit: The bit about collecting examples is not a trivial problem. Note that the successes of deep learning so far are in domains where not only the objective is well defined ("choose one of n categorical labels") but also the data associated with the objectives is easy to collect and has an obvious relation to the outputs. Say, if you want to train an image classifier to recognise images of dogs- obviously you need to collect images of dogs. This is not the case in "increase GDP" type objectives, where it is not even clear what exactly influences a country's GDP. In principle, you could feed the entire world as examples to the model, but in practice, that's just unfeasible.
- ShabbosGoy 8y ago

AndrewKemendo 8y ago

What's especially shocking about your statement is the complete lack of research you did on Ian before posting this. https://www.ianhogarth.com/about/ https://www.ianhogarth.com/about/ https://en.m.wikipedia.org/wiki/Ian_Hogarth https://en.m.wikipedia.org/wiki/Ian_Hogarth Ignoring the unsubstantiated ad hominem for a minute though, it's getting frustrating that the mere introduction of the topic of discussing possible AI futures causes immediate derision. It's as though the corporate research community, for which I'm associated, is vehemently against even discussing AGI. Whereas Barto, Bengio, Hassabis etc... are happy to discuss it in reasonable ways. The author seems to be discussing it reasonably and not making crazy kurzweilesque prognostication. Everyone at the Dartmouth Workshop wanted to create human level intelligence. Let's stop pretending that's not still the goal. OpenAI and DeepMind have that as explicit goals and if you dig into any serious AI researchers they say that's the goal vector. So where's the beef? I think it comes from a combination of being scared of overhyping AI and causing the next AI winter, and some form of gatekeeping.