3 ms·

You are saying that all top ai scientists are saying fairy tales to scare themselves if I understood correctly?

jonathankoren 3y ago
Yes. Seriously. It’s stupid talk to encourage regularity capture. If they were really afraid they were building a world ending device, they’d stop.
- femiagbabiaka 3y ago
  Oh for sure. https://en.wikipedia.org/wiki/Manhattan_Project https://en.wikipedia.org/wiki/Manhattan_Project
  - jonathankoren 3y ago
    Well that’s a bit mischaracterization of the Manhattan Project, and the views of everyone involved now isn’t it? Write a thought. You’re not clever enough for a drive by gotcha
    - femiagbabiaka 3y ago
      > Well that’s a bit mischaracterization of the Manhattan Project, and the views of everyone involved now isn’t it? Is it? The push for the bomb was an international arms race — America against Russia. The race for AGI is an international arms race — America against China. The Manhattan Project members knew that what they were doing would have terrible consequences for the world but decided to forge ahead. It’s hard to say concretely what the leaders in AGI believe right now. Ideology (and fear, and greed) can cause well meaning people to do terrible things. It does all the time. If Anthropic, OpenAI, etc. believed they had access to world ending technology they wouldn’t stop, they’d keep going so that the U.S. could have a monopoly on it. And then we’d need a chastened figure ala Oppenheimer to right the balance again.
      - qwytw 3y ago
        > The push for the bomb was an international arms race — America against Russia Was it? US (and initially UK) didn't really face any real competition at all until the war was already over and they had the bomb. The Soviets then just stole American designs and iterated on top of them.
        femiagbabiaka 3y ago
        You know that now, with the benefit of history. At the time the fear of someone else developing the bomb first was real, and the Soviet Union knew about the Manhattan project: https://www.atomicarchive.com/history/cold-war/page-9.html https://www.atomicarchive.com/history/cold-war/page-9.html.
        qwytw 3y ago
        Isn't this mainly about what happened after the war and developing then hydrogen bomb? Did anyone seriously believe during WW2 that the Nazis/Soviets could be the first to develop a nuclear weapon (I don't really know to be fair)?
        femiagbabiaka 3y ago
        A lot of it happened after the war, but the Nazis had their own nuclear program that was highly infiltrated by the CIA, and whose progress was tracked against. Considering how late Teller's mechanism for detonation was developed, the race against time was real.
Apocryphon 3y ago
The Manhattan Project physicists once feared setting the atmosphere on fire. Scientific paradigms progress with time.
- cthalupa 3y ago
  This fear seems to have been largely played up for drama. My understanding of the situation is that at one point they went 'Huh, we could potentially set off a chain reaction here. We should check out if the math adds up on that.' Then they went off and did the math and quickly found that this wouldn't happen because the amount of energy in play here was order of magnitudes lower than what would be needed for such a thing to occur and went on about their day. The only reason it's something we talk about is because of the nature of the outcome, not how seriously the physicists were in their fear.
- deleted 3y ago
  [deleted]
objektif 3y ago
Yeah kind of like how we as US ask developing countries to reduce carbon emissions.
smegger001 3y ago
Even they have grown up in a world where Frankenstein's Monster is the predominant cultural narrative for AI. most movies books shows games etc all say ai will turn on you (even though a reading of Marry Shelly's opus will tell you the creator was the monster not the creature that isnt the narrative in the publics collective subconscious believes it is). I personalmy prefer Asimov's veiw of ai in that its a tool and we dont make tools to hurt us they will be aligned with us because they designed such that their motivation will be to serve us.
- IanCal 3y ago
  > we dont make tools to hurt us We have many cases of creating things that harm us. We tore a hole in the ozone layer, filled things with lead and plastics and are facing upheaval due to climate change. > they will be aligned with us because they designed such that their motivation will be to serve us. They won't hurt us, all we asked for is paperclips. The obvious problem here is how well you get to constrain the output of an intelligence. This is not a simple problem.
  - smegger001 3y ago
    Honestly we already have paperclip maximizers they are called corporations. Instead of paperclips they are maximizing for shortterm shareholder value.
- Davidzheng 3y ago
  Can a superintelligence ever be merely a tool?
  - smegger001 3y ago
    If it has no motivation and drives of its own yeah why not. AI wont have a "psychology" anything like our own it wont feel pain it wont feel emotions it wont feel biological imparetives. All it will have is its programming /training to do what its been told. Neural nets that dont produce the right outcomes will be trained and reweighted until they do.
- 3y ago
adastra22 3y ago
Not all, or even arguably most AI researchers subscribe to The Big Scary Idea.
- arisAlexis 3y ago
  Actually the majority of the VA top current. That is Ilya, hassabis, anthropic, Bengio, Hinton. 3 top labs? 3 same views.

Yes. Seriously. It’s stupid talk to encourage regularity capture. If they were really afraid they were building a world ending device, they’d stop.

femiagbabiaka 3y ago
Oh for sure. https://en.wikipedia.org/wiki/Manhattan_Project https://en.wikipedia.org/wiki/Manhattan_Project
- jonathankoren 3y ago
  Well that’s a bit mischaracterization of the Manhattan Project, and the views of everyone involved now isn’t it? Write a thought. You’re not clever enough for a drive by gotcha
  - femiagbabiaka 3y ago
    > Well that’s a bit mischaracterization of the Manhattan Project, and the views of everyone involved now isn’t it? Is it? The push for the bomb was an international arms race — America against Russia. The race for AGI is an international arms race — America against China. The Manhattan Project members knew that what they were doing would have terrible consequences for the world but decided to forge ahead. It’s hard to say concretely what the leaders in AGI believe right now. Ideology (and fear, and greed) can cause well meaning people to do terrible things. It does all the time. If Anthropic, OpenAI, etc. believed they had access to world ending technology they wouldn’t stop, they’d keep going so that the U.S. could have a monopoly on it. And then we’d need a chastened figure ala Oppenheimer to right the balance again.
    - qwytw 3y ago
      > The push for the bomb was an international arms race — America against Russia Was it? US (and initially UK) didn't really face any real competition at all until the war was already over and they had the bomb. The Soviets then just stole American designs and iterated on top of them.
      - femiagbabiaka 3y ago
        You know that now, with the benefit of history. At the time the fear of someone else developing the bomb first was real, and the Soviet Union knew about the Manhattan project: https://www.atomicarchive.com/history/cold-war/page-9.html https://www.atomicarchive.com/history/cold-war/page-9.html.
        qwytw 3y ago
        Isn't this mainly about what happened after the war and developing then hydrogen bomb? Did anyone seriously believe during WW2 that the Nazis/Soviets could be the first to develop a nuclear weapon (I don't really know to be fair)?
        femiagbabiaka 3y ago
        A lot of it happened after the war, but the Nazis had their own nuclear program that was highly infiltrated by the CIA, and whose progress was tracked against. Considering how late Teller's mechanism for detonation was developed, the race against time was real.

Apocryphon 3y ago

The Manhattan Project physicists once feared setting the atmosphere on fire. Scientific paradigms progress with time.

cthalupa 3y ago
This fear seems to have been largely played up for drama. My understanding of the situation is that at one point they went 'Huh, we could potentially set off a chain reaction here. We should check out if the math adds up on that.' Then they went off and did the math and quickly found that this wouldn't happen because the amount of energy in play here was order of magnitudes lower than what would be needed for such a thing to occur and went on about their day. The only reason it's something we talk about is because of the nature of the outcome, not how seriously the physicists were in their fear.
deleted 3y ago
[deleted]

objektif 3y ago

Yeah kind of like how we as US ask developing countries to reduce carbon emissions.

smegger001 3y ago

Even they have grown up in a world where Frankenstein's Monster is the predominant cultural narrative for AI. most movies books shows games etc all say ai will turn on you (even though a reading of Marry Shelly's opus will tell you the creator was the monster not the creature that isnt the narrative in the publics collective subconscious believes it is). I personalmy prefer Asimov's veiw of ai in that its a tool and we dont make tools to hurt us they will be aligned with us because they designed such that their motivation will be to serve us.

IanCal 3y ago
> we dont make tools to hurt us We have many cases of creating things that harm us. We tore a hole in the ozone layer, filled things with lead and plastics and are facing upheaval due to climate change. > they will be aligned with us because they designed such that their motivation will be to serve us. They won't hurt us, all we asked for is paperclips. The obvious problem here is how well you get to constrain the output of an intelligence. This is not a simple problem.
- smegger001 3y ago
  Honestly we already have paperclip maximizers they are called corporations. Instead of paperclips they are maximizing for shortterm shareholder value.
Davidzheng 3y ago
Can a superintelligence ever be merely a tool?
- smegger001 3y ago
  If it has no motivation and drives of its own yeah why not. AI wont have a "psychology" anything like our own it wont feel pain it wont feel emotions it wont feel biological imparetives. All it will have is its programming /training to do what its been told. Neural nets that dont produce the right outcomes will be trained and reweighted until they do.
3y ago

adastra22 3y ago

Not all, or even arguably most AI researchers subscribe to The Big Scary Idea.

arisAlexis 3y ago
Actually the majority of the VA top current. That is Ilya, hassabis, anthropic, Bengio, Hinton. 3 top labs? 3 same views.