5 ms·

Disappointing. Yet another industry leader sewing public FUD for some reason. Why not bring rational discourse into the conversation around software safety and

by lxe 3y ago

Disappointing. Yet another industry leader sewing public FUD for some reason. Why not bring rational discourse into the conversation around software safety and ethics. Automation has been the driving force of industry since the industrial revolution itself. We're not new to automation, and we are certainly not new to safety of autonomous systems. AI is no different.

itake 3y ago
I suspect there is more to it than what has been published. These are smart people that appear (to us) acting irrationally.
- whimsicalism 3y ago
  Irrationally how? What is so obviously irrational about what is being done? I don't see it.
  - itake 3y ago
    I don't want to speculate his reasons, but I don't see how leaving his influential role at a top AI company (Google) accomplishes the goals written about in the paper.
1attice 3y ago
Prove that "AI is no different." Its creators appear to differ with you on this point. The burden of proof is thus yours.
lifeinthevoid 3y ago
What are your credentials in the field if I may ask?
- lxe 3y ago
  Senior Hacker News Commenter
EVa5I7bHFq9mnYK 3y ago
It is different. The most powerful of today's machines has a red stop button. But if a machine becomes smarter than us, it could create a copy of itself without such button, so we lose control and will be quickly overpowered.
- janalsncm 3y ago
  There’s an argument that we’ve gone past that point already. Yes, Microsoft can theoretically take their Bing GPT-4 program offline and turn it off, but they just invested $10B in it and they don’t want to. In fact a corporation can be thought of as an AGI itself, just made up of humans. Again, we can take Microsoft offline but we don’t want to. I guess my point is that the most likely scenario for AGI that looks more like AGI isn’t that we won’t be able to take it down but we won’t want to.
  - whimsicalism 3y ago
    I see lots of people pointing to what is "more likely" or "more realistic." I'm not sure where everyone got these strong priors on the consequences of 'intelligent' machines from, did I miss some memo?
    - janalsncm 3y ago
      Do you disagree with my first paragraph?
capableweb 3y ago
One major difference between now and then is that now automation is starting to look and behave in a way that can be confused with a human. Most, if not all, comments generated by machines before LLMs could be identified as such, while now it's going to get harder and harder to detect properly. Quick evaluation: did a human write this comment or did I use GPT-4 to write this comment by just providing what meaning I wanted to convey? The answer is f3bd3abcb05c3a362362a17f690d73aa7df15eb2acf4eb5bf8a5d39d07bae216 (sha256sum)
oldstrangers 3y ago
Yeah, what could we possibly hope to learn from the "The Godfather of A.I." about the protentional dangers of A.I. Maybe... they're better positioned to opine on this topic than you are?
- ActorNightly 3y ago
  The issue is that the ML stuff is so bleeding edge and paradigm breaking that no set of prior credentials is really a valid mark for being right or wrong about this, especially those with traditional college research background. The best way to describe this issue is with a hypothetical scenario: A well respected Ph.D Physicist comes out with a paper that describes a warp drive that can be used to travel faster than light. He hands the paper to very talented multi disciplinary engineer that has built everything from small micro circuits to spaceships. Engineer says: "Ok this is cool, what do I need to do to start building this?". Researches says "Ok so first, you need to find some negative mass and gather it". Engineer says: "How do I do that"? Researcher answers: "I dont know, but once you do, find a way to arrange it into a ring around the spaceship". Its the same story with AI. Just because someone has research experience doesn't mean that he/she knows how technology will evolve in the real world. There may be theories floating around about how AI development could cause danger, but without real concrete paths of what has to happen for those theories to come true - and without those paths, those theories remain theories.
  - oldstrangers 3y ago
    Alternatively, we can draw a more fitting parallel to Robert Oppenheimer, who, upon recognizing the devastating potential of his creation, dedicated himself to halting the spread of nuclear weapons worldwide.
    - ActorNightly 3y ago
      Robert Oppenheimer knew the entire domain space of nuclear weapons. Current researchers don't. Its not like future neural networks are just going to be stacks and stacks of transformers on top of each other.
Simon321 3y ago
Too bad you're being downvoted because you're making very good points. All the counter arguments are arguments from authority also.

by lxe 3y ago

itake 3y ago

I suspect there is more to it than what has been published. These are smart people that appear (to us) acting irrationally.

whimsicalism 3y ago
Irrationally how? What is so obviously irrational about what is being done? I don't see it.
- itake 3y ago
  I don't want to speculate his reasons, but I don't see how leaving his influential role at a top AI company (Google) accomplishes the goals written about in the paper.

1attice 3y ago

Prove that "AI is no different." Its creators appear to differ with you on this point. The burden of proof is thus yours.

lifeinthevoid 3y ago

What are your credentials in the field if I may ask?

lxe 3y ago
Senior Hacker News Commenter

EVa5I7bHFq9mnYK 3y ago

It is different. The most powerful of today's machines has a red stop button. But if a machine becomes smarter than us, it could create a copy of itself without such button, so we lose control and will be quickly overpowered.

janalsncm 3y ago
There’s an argument that we’ve gone past that point already. Yes, Microsoft can theoretically take their Bing GPT-4 program offline and turn it off, but they just invested $10B in it and they don’t want to. In fact a corporation can be thought of as an AGI itself, just made up of humans. Again, we can take Microsoft offline but we don’t want to. I guess my point is that the most likely scenario for AGI that looks more like AGI isn’t that we won’t be able to take it down but we won’t want to.
- whimsicalism 3y ago
  I see lots of people pointing to what is "more likely" or "more realistic." I'm not sure where everyone got these strong priors on the consequences of 'intelligent' machines from, did I miss some memo?
  - janalsncm 3y ago
    Do you disagree with my first paragraph?

capableweb 3y ago

One major difference between now and then is that now automation is starting to look and behave in a way that can be confused with a human. Most, if not all, comments generated by machines before LLMs could be identified as such, while now it's going to get harder and harder to detect properly. Quick evaluation: did a human write this comment or did I use GPT-4 to write this comment by just providing what meaning I wanted to convey? The answer is f3bd3abcb05c3a362362a17f690d73aa7df15eb2acf4eb5bf8a5d39d07bae216 (sha256sum)

oldstrangers 3y ago

Yeah, what could we possibly hope to learn from the "The Godfather of A.I." about the protentional dangers of A.I. Maybe... they're better positioned to opine on this topic than you are?

ActorNightly 3y ago
The issue is that the ML stuff is so bleeding edge and paradigm breaking that no set of prior credentials is really a valid mark for being right or wrong about this, especially those with traditional college research background. The best way to describe this issue is with a hypothetical scenario: A well respected Ph.D Physicist comes out with a paper that describes a warp drive that can be used to travel faster than light. He hands the paper to very talented multi disciplinary engineer that has built everything from small micro circuits to spaceships. Engineer says: "Ok this is cool, what do I need to do to start building this?". Researches says "Ok so first, you need to find some negative mass and gather it". Engineer says: "How do I do that"? Researcher answers: "I dont know, but once you do, find a way to arrange it into a ring around the spaceship". Its the same story with AI. Just because someone has research experience doesn't mean that he/she knows how technology will evolve in the real world. There may be theories floating around about how AI development could cause danger, but without real concrete paths of what has to happen for those theories to come true - and without those paths, those theories remain theories.
- oldstrangers 3y ago
  Alternatively, we can draw a more fitting parallel to Robert Oppenheimer, who, upon recognizing the devastating potential of his creation, dedicated himself to halting the spread of nuclear weapons worldwide.
  - ActorNightly 3y ago
    Robert Oppenheimer knew the entire domain space of nuclear weapons. Current researchers don't. Its not like future neural networks are just going to be stacks and stacks of transformers on top of each other.

Simon321 3y ago

Too bad you're being downvoted because you're making very good points. All the counter arguments are arguments from authority also.