6 ms·

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug. I wi

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug. I wish Eliezer and others would set aside meta-AI (dire predictions and worries about AI, the coming singularity, etc.) and concentrate on the problem of creating AI Guess there's no money in that. If only someone would pull the plug on this nonsense...

gizmo 17y ago
Oh no, somebody is doing research in an as of yet unexplored area! That must be a humongous waste of time! Pull the plug before too many papers have been written! Before I read Eliezer's work I thought the design an AI first, think about details & security afterward was a viable strategy. Eliezer illustrates how such an approach can go very wrong. Valuable research, if you ask me. PS: pulling the plug misses the point. The point is that a transhuman AI can convince the GK that the GK wants to let the AI go free. The AI can offer a big bribe (money/power/reputation) and argue that eventually somebody will let an AI go free anyway, so why not stop the futile resistance, accept the bribe, and move on?
- makmanalp 17y ago
  It's not a waste of time, but so far Eliezer has demonstrated nothing. Why would you do research and announce the results to the public if you're not going to also announce the steps to make it reproducible and confirmable? Maybe announcing the steps themselves renders the method unusable?
- wlievens 17y ago
  Now, let me play devil's advocate here, as I do think that Eliezer writes interesting stuff; but what is he contributing to the field of AI beyond what movies like Terminator, Space Odyssey or Eagle Eye also illustrate (AI takes over humans)?
asciilifeform 17y ago
> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity. Give us back the pre-2001 Eliezer. The one who wrote code. Because safety is not safe. (http://lesswrong.com/lw/10n/why_safety_is_not_safe/ http://lesswrong.com/lw/10n/why_safety_is_not_safe/)
- andreyf 17y ago
  Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safety 200 years ago.
  - MarkPNeyer 17y ago
    An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.
    - asciilifeform 17y ago
      > a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust the non-renewable resources to which our civilization is addicted and regress into permanent barbarism. Give me "death by AI" any day of the week, over that.
      - MarkPNeyer 17y ago
jerf 17y ago
Waiting until after someone blithely writes a superhuman AI with its primary drive set to something like manufacturing as many cars as possible (sensible in a factory, somewhat less sensible in the real world), followed by it escaping out into the world and turning the entire resources of the planet to car manufacturing at all costs (including those pesky humans who do not seem to wish to be turned into cars, too bad for them), to consider the problems of powerful AI seems like a really bad idea. I would consider that sentence as a candidate for "understatement of the year". (Don't get too caught up in the "cars" part. What the superhuman AI intends to do hardly matters in the end; the absurdity is part of my point and deliberate. Only a vanishing fraction of possible primary motivations end up with happy humans on the other end.)

gizmo 17y ago

Oh no, somebody is doing research in an as of yet unexplored area! That must be a humongous waste of time! Pull the plug before too many papers have been written! Before I read Eliezer's work I thought the design an AI first, think about details & security afterward was a viable strategy. Eliezer illustrates how such an approach can go very wrong. Valuable research, if you ask me. PS: pulling the plug misses the point. The point is that a transhuman AI can convince the GK that the GK wants to let the AI go free. The AI can offer a big bribe (money/power/reputation) and argue that eventually somebody will let an AI go free anyway, so why not stop the futile resistance, accept the bribe, and move on?

makmanalp 17y ago
It's not a waste of time, but so far Eliezer has demonstrated nothing. Why would you do research and announce the results to the public if you're not going to also announce the steps to make it reproducible and confirmable? Maybe announcing the steps themselves renders the method unusable?
wlievens 17y ago
Now, let me play devil's advocate here, as I do think that Eliezer writes interesting stuff; but what is he contributing to the field of AI beyond what movies like Terminator, Space Odyssey or Eagle Eye also illustrate (AI takes over humans)?

asciilifeform 17y ago

> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity. Give us back the pre-2001 Eliezer. The one who wrote code. Because safety is not safe. (http://lesswrong.com/lw/10n/why_safety_is_not_safe/ http://lesswrong.com/lw/10n/why_safety_is_not_safe/)

andreyf 17y ago
Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safety 200 years ago.
- MarkPNeyer 17y ago
  An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.
  - asciilifeform 17y ago
    > a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust the non-renewable resources to which our civilization is addicted and regress into permanent barbarism. Give me "death by AI" any day of the week, over that.
    - MarkPNeyer 17y ago

jerf 17y ago

Waiting until after someone blithely writes a superhuman AI with its primary drive set to something like manufacturing as many cars as possible (sensible in a factory, somewhat less sensible in the real world), followed by it escaping out into the world and turning the entire resources of the planet to car manufacturing at all costs (including those pesky humans who do not seem to wish to be turned into cars, too bad for them), to consider the problems of powerful AI seems like a really bad idea. I would consider that sentence as a candidate for "understatement of the year". (Don't get too caught up in the "cars" part. What the superhuman AI intends to do hardly matters in the end; the absurdity is part of my point and deliberate. Only a vanishing fraction of possible primary motivations end up with happy humans on the other end.)