6 ms·

You could make basically the same argument for why humans will never invent heavier-than-air flight - because avian flight 'evolved based on a need to survive a

by feral 11y ago

You could make basically the same argument for why humans will never invent heavier-than-air flight - because avian flight 'evolved based on a need to survive and reproduce', and 'artificial flight doesn't have darwinian pressures'. But humans did make artificial flying machines. I don't mean to be unkind, but your argument doesn't have close to the level of certainly that we should bet our species on. No one knows where the roadblocks to AGI are, or what the timelines might be; but there is what seems like huge progress happening recently in an area which might eventually lead there. While no one knows the path, many intelligent people have thought about this without finding any theoretical roadblocks. Please don't publicly dismiss the concerns as 'science fiction' without a little more thought.

andreyf 11y ago
I'm not saying that HGI is impossible, but that the "risk scenarios" are complete fiction. In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. This will change the questions of risk into something that's tangible and addressable.
- feral 11y ago
  On Ada Lovelace's 200 year birthday, we might remember her quote: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform." She was very smart, but she was wrong about that. >In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? If so, I disagree with your assertion: it is not obvious that those things have to be done before a human level AGI could be made. It might be a good idea to do so, but its not clear that they are a necessary requirement. Some people would say this is why AGI is dangerous. Maybe there's a path to bootstrapping an AGI, where we build something from relatively simple bits we can understand, that then becomes very much more complex than we can understand. This happens all the time: for example, we get very complex classification behaviour from deep nets: [simple pieces] + [lots of data] doing something more complex than we have explicitly trained them to. If we were convinced that "In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior" was a necessary thing to build before we could build a HGI, people would be less worried. But its not; and that's partially why people are worried. In fact, some of the people working on AI risk could be said to be basically racing to understand these things before other folk build an AGI.
  - joe_the_user 11y ago
    You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. That may or may not be ethics as such but if something we make on purpose, we'd need to understand it. And considering it's difficulties, making it on purpose seems necessary, contrary to movies etc.
    - feral 11y ago
      Yes - but no one provided any argument for that. Instead, andreyf relied on it as a fact. I pointed out we can't assume that. As such: its reasonable to fear AGI coming before we have that stuff sorted out. Hence, as such: there is a real risk here, which the safety folks are trying to mitigate; hence they aren't worried about a threat scenario easily dismissable by andreyf's argument. >It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. Ie. If you, or anyone had convincingly argued this, then it would be reasonable to dismiss (or at least greatly reduce) the concerns of the safety folk. But no one has, so its not reasonable to.
      - andreyf 11y ago
        It is absolutely reasonable to fear how ML may be misused by a human, and those concerns fall mostly in the realm of political theory and economics. My claim is that I have seen no evidence that it is reasonable to fear a fictional self-aware autonomous AI (what I call HGI). The evidence people point to seems to have a very tenuous relationship with the reality of AI research. Once we have even an inkling of an idea of how one might approach creating an artificial HGI, we can discuss how we can prevent it from being evil and taking over the world. Until there's actually a proposal for how one might construct one, though, any such discussion is worse than useless. How can you talk about the risk of something when you literally have no idea what it is you are describing or how it's built, embodied with magical properties imagined by science fiction authors? To me, this makes about as much sense as talking about stopping aliens which could very well exist and could very well wipe out our civilization, i.e. great subject for wrapping up a party or going to the movies, but not exactly material for peer reviewed journals.
joe_the_user 11y ago
You could make basically the same argument for why humans will never invent heavier-than-air flight You're not getting it. The effective argument of the danger seer is akin to "humans will learn to fly but then our cities will be covered with feathers" - Humans have incentive to produce flexible, learning, language-understanding machines. Humans don't an incentive to produce a thing that will suddenly go rogue and decide to kill us. It is entirely reasonable to argue that the tendency of humans themselves to go rogue and decide kill other humans comes from the evolutionary processes that produced humans rather than from the quality of human intelligence. We can see other relatively intelligent animals that live with in more (and in less) harmony than humans. No reason intelligence implies entity X won't fully cooperate with the people who intentionally engineered it. I would argue that we human imagine that AGIs would have the same "downsides" as ourselves simply because we don't have any conception of intelligent things besides ourselves. Maybe a carefully human-engineered intelligence would for reasons unknown turn into the things we see on movies. Maybe the next generation of supercolider will create an subatomic chain reaction that destroys the earth. But it's premature to worry about these particular hypothetical, especially given humans with ordinary AIs do pose problems.
- shpx 11y ago
  We also don't have an incentive to write programs that grant access to people who know a special incantation like '() { :; };' and yet we do. Point being we can fuck this up very easily, like any other bit of software. I myself am on fence, on the one hand it is a bit like worrying about over population on mars, and there really is nothing we can productively do right now to research AI safety, the best thing we have are convolutional long/short term memory nets, and there's not much you can say about it. But at the same time worrying is a great move if the alternative is to just wait until we can do something.
  - joe_the_user 11y ago
    We unintentionally produce programs while fail or produce small variations on their original purpose - give control to X person rather than Y person. We haven't unintentionally produced a program that had a whole range of unexpected behaviors. An intelligence having human-like "survival instincts" and etc would be a big variation on a tool.

by feral 11y ago

andreyf 11y ago

I'm not saying that HGI is impossible, but that the "risk scenarios" are complete fiction. In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. This will change the questions of risk into something that's tangible and addressable.

feral 11y ago
On Ada Lovelace's 200 year birthday, we might remember her quote: "The Analytical Engine has no pretensions to originate anything. It can do whatever we know how to order it to perform." She was very smart, but she was wrong about that. >In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior. You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? If so, I disagree with your assertion: it is not obvious that those things have to be done before a human level AGI could be made. It might be a good idea to do so, but its not clear that they are a necessary requirement. Some people would say this is why AGI is dangerous. Maybe there's a path to bootstrapping an AGI, where we build something from relatively simple bits we can understand, that then becomes very much more complex than we can understand. This happens all the time: for example, we get very complex classification behaviour from deep nets: [simple pieces] + [lots of data] doing something more complex than we have explicitly trained them to. If we were convinced that "In our journey to achieving HGI, we must first achieve an understanding of HGI ethics, motivations, and behavior" was a necessary thing to build before we could build a HGI, people would be less worried. But its not; and that's partially why people are worried. In fact, some of the people working on AI risk could be said to be basically racing to understand these things before other folk build an AGI.
- joe_the_user 11y ago
  You mean that it's impossible to make a human level general intelligence ('HGI', to use your term), without understanding those things? It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. That may or may not be ethics as such but if something we make on purpose, we'd need to understand it. And considering it's difficulties, making it on purpose seems necessary, contrary to movies etc.
  - feral 11y ago
    Yes - but no one provided any argument for that. Instead, andreyf relied on it as a fact. I pointed out we can't assume that. As such: its reasonable to fear AGI coming before we have that stuff sorted out. Hence, as such: there is a real risk here, which the safety folks are trying to mitigate; hence they aren't worried about a threat scenario easily dismissable by andreyf's argument. >It is entirely logical to argue that a high level intelligence could not be constructed without a broad and deep understanding of it's motivations and behaviors. Ie. If you, or anyone had convincingly argued this, then it would be reasonable to dismiss (or at least greatly reduce) the concerns of the safety folk. But no one has, so its not reasonable to.
    - andreyf 11y ago
      It is absolutely reasonable to fear how ML may be misused by a human, and those concerns fall mostly in the realm of political theory and economics. My claim is that I have seen no evidence that it is reasonable to fear a fictional self-aware autonomous AI (what I call HGI). The evidence people point to seems to have a very tenuous relationship with the reality of AI research. Once we have even an inkling of an idea of how one might approach creating an artificial HGI, we can discuss how we can prevent it from being evil and taking over the world. Until there's actually a proposal for how one might construct one, though, any such discussion is worse than useless. How can you talk about the risk of something when you literally have no idea what it is you are describing or how it's built, embodied with magical properties imagined by science fiction authors? To me, this makes about as much sense as talking about stopping aliens which could very well exist and could very well wipe out our civilization, i.e. great subject for wrapping up a party or going to the movies, but not exactly material for peer reviewed journals.

joe_the_user 11y ago

You could make basically the same argument for why humans will never invent heavier-than-air flight You're not getting it. The effective argument of the danger seer is akin to "humans will learn to fly but then our cities will be covered with feathers" - Humans have incentive to produce flexible, learning, language-understanding machines. Humans don't an incentive to produce a thing that will suddenly go rogue and decide to kill us. It is entirely reasonable to argue that the tendency of humans themselves to go rogue and decide kill other humans comes from the evolutionary processes that produced humans rather than from the quality of human intelligence. We can see other relatively intelligent animals that live with in more (and in less) harmony than humans. No reason intelligence implies entity X won't fully cooperate with the people who intentionally engineered it. I would argue that we human imagine that AGIs would have the same "downsides" as ourselves simply because we don't have any conception of intelligent things besides ourselves. Maybe a carefully human-engineered intelligence would for reasons unknown turn into the things we see on movies. Maybe the next generation of supercolider will create an subatomic chain reaction that destroys the earth. But it's premature to worry about these particular hypothetical, especially given humans with ordinary AIs do pose problems.

shpx 11y ago
We also don't have an incentive to write programs that grant access to people who know a special incantation like '() { :; };' and yet we do. Point being we can fuck this up very easily, like any other bit of software. I myself am on fence, on the one hand it is a bit like worrying about over population on mars, and there really is nothing we can productively do right now to research AI safety, the best thing we have are convolutional long/short term memory nets, and there's not much you can say about it. But at the same time worrying is a great move if the alternative is to just wait until we can do something.
- joe_the_user 11y ago
  We unintentionally produce programs while fail or produce small variations on their original purpose - give control to X person rather than Y person. We haven't unintentionally produced a program that had a whole range of unexpected behaviors. An intelligence having human-like "survival instincts" and etc would be a big variation on a tool.