3 ms·

Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. Both the definition of AGI given by the company and by AI aliment/

Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. Both the definition of AGI given by the company and by AI aliment/safety researchers is similar. AI safety people trust it is dangerous. let me go continue using a nuclear bomb as a metaphor. if we don't know whether building nuclear bomb is possible, but some companies declare they have existing progress on creating this new bomb... the danger of nuclear bomb is obvious, because it is designed as a bomb. companies are trying to build the AGI which is similar to the dangerous AGI in AI safety researchers' prediction. the dangers are obvious, too.

kolektiv 3y ago
They declare that - but I could also declare I'm trying to build a nuclear bomb (n.b. I'm not). Whether people are likely to try and stop me, or try and apply some legal non-proliferation framework, is partly influenced by whether they believe what I'm claiming is realistic (it's not - I have a workshop, but no fissile material). Nobody gets too worried about me doing something which would be awful, but the general consensus is that I won't achieve. Until a company gives some credible evidence they're close to AGI... (And companies have millions/billions of reasons to claim they are when they're not, so scepticism is warranted).
- psychoslave 3y ago
  All good points. Now playing viled advocate: building a nuclear bomb in my basement was very difficult, I admit. But since I already have my spywares installed everywhere, the moment a dude come with an AGI, it will directly be shared to all my follow hackers through bittorent, eDonkey, Hyphanet, Gnunet, Kad and Tor, just to name a few.
- Snow_Falls 3y ago
  It is surely better to have regulation now than scramble to catch up if ago is possible.
  - daynthelife 3y ago
    What about nation states? Do you really think the US military will avoid working towards AGI if they think it would give them a tactical advantage? Or the CCP? Or North Korea? Personally, if AGI gets developed, I’d rather the first iteration be in the hands of someone who doesn’t also have access to nuclear weapons.
    - Snow_Falls 3y ago
      Why wouldn't the government just intasntly take it? Hell, why wouldn't the corporations just sell it to the highest bidder?
      - daynthelife 3y ago
        They may well. But if we ban research altogether, the only ones with access will be governments. At least with an open system there would be some competition.
- melagonster 3y ago
  So do we just pray AGI is impossible? Companies are trying and spend many money to make it possible in this moment.
- jamilton 3y ago
  I think it's reasonable to at least act as if they really are trying to do what they say they're trying to do. Like with the nuclear bomb situation - I think it would be reasonable for someone to try to stop you from building a nuclear bomb, or check if you really were, even if they had no idea how you could have gotten the materials. Because it would be really bad if you did. I think people would be worried about you trying to do something awful even if there was low confidence it was possible. They wouldn't be as worried as they would be if they thought you were more capable, but still worried. So I guess that comes down to whether you think companies saying they want to make AGI are more like a toddler, a teenager, you, a ballistics engineer who owns a uranium mine, Lockheed-Martin, or a nation-state trying to make a nuclear bomb. My understanding is that people who are concerned about AI x-risk are largely in the teenager to Lockheed-Martin range (e.g., small eventual risk to large imminent risk), while I assume you think it's more in the toddler range (no risk at all for a long time)
nerdbert 3y ago
> Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. They do declare it, but nobody has even come up with a plausible path from where we are today, to anything like AGI. At this point they might as well declare that they're trying to build time machines.
- melagonster 3y ago
  yes, but why we even give them chances? when they know their ultimately purpose is dangerous?

kolektiv 3y ago

They declare that - but I could also declare I'm trying to build a nuclear bomb (n.b. I'm not). Whether people are likely to try and stop me, or try and apply some legal non-proliferation framework, is partly influenced by whether they believe what I'm claiming is realistic (it's not - I have a workshop, but no fissile material). Nobody gets too worried about me doing something which would be awful, but the general consensus is that I won't achieve. Until a company gives some credible evidence they're close to AGI... (And companies have millions/billions of reasons to claim they are when they're not, so scepticism is warranted).

psychoslave 3y ago
All good points. Now playing viled advocate: building a nuclear bomb in my basement was very difficult, I admit. But since I already have my spywares installed everywhere, the moment a dude come with an AGI, it will directly be shared to all my follow hackers through bittorent, eDonkey, Hyphanet, Gnunet, Kad and Tor, just to name a few.
Snow_Falls 3y ago
It is surely better to have regulation now than scramble to catch up if ago is possible.
- daynthelife 3y ago
  What about nation states? Do you really think the US military will avoid working towards AGI if they think it would give them a tactical advantage? Or the CCP? Or North Korea? Personally, if AGI gets developed, I’d rather the first iteration be in the hands of someone who doesn’t also have access to nuclear weapons.
  - Snow_Falls 3y ago
    Why wouldn't the government just intasntly take it? Hell, why wouldn't the corporations just sell it to the highest bidder?
    - daynthelife 3y ago
      They may well. But if we ban research altogether, the only ones with access will be governments. At least with an open system there would be some competition.
melagonster 3y ago
So do we just pray AGI is impossible? Companies are trying and spend many money to make it possible in this moment.
jamilton 3y ago
I think it's reasonable to at least act as if they really are trying to do what they say they're trying to do. Like with the nuclear bomb situation - I think it would be reasonable for someone to try to stop you from building a nuclear bomb, or check if you really were, even if they had no idea how you could have gotten the materials. Because it would be really bad if you did. I think people would be worried about you trying to do something awful even if there was low confidence it was possible. They wouldn't be as worried as they would be if they thought you were more capable, but still worried. So I guess that comes down to whether you think companies saying they want to make AGI are more like a toddler, a teenager, you, a ballistics engineer who owns a uranium mine, Lockheed-Martin, or a nation-state trying to make a nuclear bomb. My understanding is that people who are concerned about AI x-risk are largely in the teenager to Lockheed-Martin range (e.g., small eventual risk to large imminent risk), while I assume you think it's more in the toddler range (no risk at all for a long time)

nerdbert 3y ago

> Companies declare that they are trying to build better AI, and the ultimately purpose is AGI. They do declare it, but nobody has even come up with a plausible path from where we are today, to anything like AGI. At this point they might as well declare that they're trying to build time machines.

melagonster 3y ago
yes, but why we even give them chances? when they know their ultimately purpose is dangerous?