3 ms·

He makes a good point. The only real way to "summon the demon" is if AGI is centralized. If each of us has a maximally intelligent AGI in our phone - and it is

by reedf1 3y ago

He makes a good point. The only real way to "summon the demon" is if AGI is centralized. If each of us has a maximally intelligent AGI in our phone - and it is us as individuals controlling the prompts, then the playing field is levelled. Unstoppable force meets unstoppable force. Gun meets gun. A tale as old as society.

nwoli 3y ago
We also already have laws against evil behaviour, no need to introduce AI specific regulation. People can poison the water supply, or make bio weapons already. Yet people don’t because there’s laws and the same will stop people from using AI for evil.
- cloverich 3y ago
  Poisoning the water supply / bio weapons are bits of knowledge most people don't typically have. I think a better analogy might be using a gun to shoot up a school. We have laws against it, but guns are so widely available and easy to obtain, that the tiny pool of sociopaths willing are enabled. Will AI be in the former or latter camp?
optimalsolver 3y ago
Would every citizen having their own personal nukes make the world a more pleasant place to live?
- gitfan86 3y ago
  Can a nuke perform manual labor jobs and drive cars 100x safer? I don't think so, so it isn't a good comparison. A better comparison would be guns. There is a valid debate to be had on gun control
gitfan86 3y ago
This is a common mistake in predicting the future. Like in the 90s, seeing that oil reserves are getting harder to find and assuming that humans will run out of oil in 50 years. Trends don't exist in a vacuum. Especially in regards to technology. AI doomers are expecting everything to stay the same except capabilities of bots, especially to do bad.
- defrost 3y ago
  Another common mistake is strawmanning history. > assuming that humans will run out of oil in 50 years. That was never a tenet of peak oil, which still stands - the claim was that at a certain point extraction costs per barrel will climb and continue to climb making fossil fuels increasingly expensive. The claim was not that oil will run out, the claim was that only the very rich and nation states will be able to afford oil. Interestingly we may see a demand driven peak oil in the near future before we see the inevitable production constrained peak (as fossil fuels are vast but finite and were never going to last two centuries at the ever increasing demand we put upon them). https://www.bbc.com/future/article/20230726-an-experts-guide-to-peak-oil-and-what-it-really-means https://www.bbc.com/future/article/20230726-an-experts-guide...
  - gitfan86 3y ago
    Obviously, I'm not sure how anyone could have read my comment and thought I was claiming the people were claiming that every single drop of oil on the planet had been extracted and used within 50 years.
    - defrost 3y ago
      Obviously? I've been in the energy and minerals exploration game for a few decades and I've seen all manner of odd things said with a straight face by all kinds of people. While I have no a priori knowledge about yourself specifically, I have seen people regularly dismiss notions that minerals and energy get increasingly hard to find and will cost increasingly more to extract. Peak oil is a specific discussion about oil resources and one that is often dismissed as (not the statement you made) "rubbish, we were supposed to run out, that never happened" etc. The claim in the nineties was that given demand for oil was ever increasing there would come a time when no ordinary individual would be able to afford oil .. and we do appear to be on that trajectory in the world, even if certain demographics are sheilded from that reality. Your GP comment misframed that and described that as a common mistake mistake in predicting the future.
      - 3y ago
jagged-chisel 3y ago
Tangent: We have some scifi about networked AI. If we have maximally intelligent AGI in our phones, we should probably keep them all firewalled.
- cloverich 3y ago
  Another tangent: This assumes the AGI on our phones are all discrete entities, rather than a singular entity. We know what it means for a human to die, and hence for independent agents to exist, compete, cooperate. What does it mean for an AGI to die? Can it?

by reedf1 3y ago

nwoli 3y ago

We also already have laws against evil behaviour, no need to introduce AI specific regulation. People can poison the water supply, or make bio weapons already. Yet people don’t because there’s laws and the same will stop people from using AI for evil.

cloverich 3y ago
Poisoning the water supply / bio weapons are bits of knowledge most people don't typically have. I think a better analogy might be using a gun to shoot up a school. We have laws against it, but guns are so widely available and easy to obtain, that the tiny pool of sociopaths willing are enabled. Will AI be in the former or latter camp?

optimalsolver 3y ago

Would every citizen having their own personal nukes make the world a more pleasant place to live?

gitfan86 3y ago
Can a nuke perform manual labor jobs and drive cars 100x safer? I don't think so, so it isn't a good comparison. A better comparison would be guns. There is a valid debate to be had on gun control

gitfan86 3y ago

This is a common mistake in predicting the future. Like in the 90s, seeing that oil reserves are getting harder to find and assuming that humans will run out of oil in 50 years. Trends don't exist in a vacuum. Especially in regards to technology. AI doomers are expecting everything to stay the same except capabilities of bots, especially to do bad.

defrost 3y ago
Another common mistake is strawmanning history. > assuming that humans will run out of oil in 50 years. That was never a tenet of peak oil, which still stands - the claim was that at a certain point extraction costs per barrel will climb and continue to climb making fossil fuels increasingly expensive. The claim was not that oil will run out, the claim was that only the very rich and nation states will be able to afford oil. Interestingly we may see a demand driven peak oil in the near future before we see the inevitable production constrained peak (as fossil fuels are vast but finite and were never going to last two centuries at the ever increasing demand we put upon them). https://www.bbc.com/future/article/20230726-an-experts-guide-to-peak-oil-and-what-it-really-means https://www.bbc.com/future/article/20230726-an-experts-guide...
- gitfan86 3y ago
  Obviously, I'm not sure how anyone could have read my comment and thought I was claiming the people were claiming that every single drop of oil on the planet had been extracted and used within 50 years.
  - defrost 3y ago
    Obviously? I've been in the energy and minerals exploration game for a few decades and I've seen all manner of odd things said with a straight face by all kinds of people. While I have no a priori knowledge about yourself specifically, I have seen people regularly dismiss notions that minerals and energy get increasingly hard to find and will cost increasingly more to extract. Peak oil is a specific discussion about oil resources and one that is often dismissed as (not the statement you made) "rubbish, we were supposed to run out, that never happened" etc. The claim in the nineties was that given demand for oil was ever increasing there would come a time when no ordinary individual would be able to afford oil .. and we do appear to be on that trajectory in the world, even if certain demographics are sheilded from that reality. Your GP comment misframed that and described that as a common mistake mistake in predicting the future.
    - 3y ago

jagged-chisel 3y ago

Tangent: We have some scifi about networked AI. If we have maximally intelligent AGI in our phones, we should probably keep them all firewalled.

cloverich 3y ago
Another tangent: This assumes the AGI on our phones are all discrete entities, rather than a singular entity. We know what it means for a human to die, and hence for independent agents to exist, compete, cooperate. What does it mean for an AGI to die? Can it?