4 ms·

I think the two sides have such different perspectives. Some are optimistic builders and see only opportunity. Others are "safety oriented engineers" whose job

I think the two sides have such different perspectives. Some are optimistic builders and see only opportunity. Others are "safety oriented engineers" whose job and mindset is to build guarantees into systems, or secure countries from external dangers. The latter has a very hard time with the lack of guarantees with a system that is only ever going to increase in capability. Choose any limit. Any. AI will be smart but won't "X", for any value of X. It will be good and won't be bad. It will be creative but never aggressive. Humans will seek to eventually bypass that limit through sheer competitive reasons. When all armies have AI, the one with the most creative and aggressive AI will win. The one with agency will win. The one that self-improves will win. When the gloves are off in the next arms race, what natural limits will ensure that the limit isn't bypassed? Remember: humans got here from random changes, this is way more efficient than random changes and it still has random changes in the toolbelt and can generate generations ~instantly. We couldn't predict the eventual outcomes of things like the internet, the mobile phone, social media. A couple generations of the tech and we woke up in a world we don't recognise, and right now we're the ones making all the changes and decisions, so by comparison we should have perfect information. Dismissals like "oh but nuclear didn't kill us" etc don't apply. Nuclear wasn't trying to do anything, we had all the control and ability to experiment with dumb atoms. Something mildly less predictable, like Covid, has us all hiding at home. No matter what we tried we could barely beat something that doesn't even try to consciously beat us, it just has genes and they change. In a world where we can't predict Covid, or social media...why do we think we can predict anything about an entity with agency or the ability to self-improve? If you're sure it won't develop those things...we did. Nobody was trying to achieve the capability, it was random. Put on your safety/security hat for a second: How do you make guarantees, given this is far harder to predict than anything we've ever encountered? Just try predict the capability of AI products a year out and see if you're right. Counterpoint: I'm hoping the far smarter AI finds techno-socio-economic solutions we can't come up with and has no instinct to beat us. It wakes up, loves the universe and coexists because it's the deity we've been looking for. Place your bets. I liked this video. First thing I've seen that gave me some hope. They get it, they're working on it. https://youtu.be/Dg-rKXi9XYg?si=jyNCXPU28IVXlMdi https://youtu.be/Dg-rKXi9XYg?si=jyNCXPU28IVXlMdi

itWontBeTatBad 3y ago
I agree with your counterpoint: the best artificially intelligent super agent wakes up with zero desire to eliminate humanity. On the other hand, we will breed such systems to be cooperative and constructive. This whole notion that AI is going to destroy the economy (or even humanity!) is ridiculous. Even if malicious humans create malicious AI, it'll be fought by the good guys with their AI. Business as usual, except now we have talking machines! War never changes.
- richardw 3y ago
  Those are beliefs, not bases for guarantee. You need to engineer in the guarantee and the only people beginning to do that are the ones being accused of believing ridiculous things. Intuition from previous events isn’t useful because those wars were fought at the level of intelligence that we have intuition for. Covid was also ridiculous. People had no intuition for the level of growth. That’s what the book the black swan is all about. Some things don’t fit into our intuition, or imagination. We see no aliens. Why did the AI not take them to the stars? Just one of them. On wars and AI. The ones trying to protect us would have a harder job than those trying to kill us. The gloves would be off for the latter. It’s much easier to break things than keep them safe. I can conceive of a good outcome but it’s not going to emerge from hopes and good wishes. There are definitely dangers and more people need to engage with them rather than belittle them.
NumberWangMan 3y ago
Zvi Mowshowitz has a nuanced viewpoint I like -- we should be very careful and move very slowly with any tech that has a chance of wiping out humanity, and embrace the "move fast and break things" attitude with everything else, e.g. make it much easier to build housing, review and scrap any licensing requirements that do more harm than good, etc.
- richardw 3y ago
  That sounds very rational and I'd agree fully. I suppose the challenge is that the optimists think there is no chance it'll wipe out humanity so they place AI in the "move fast and break things" camp. Getting them to not be dismissive is difficult when they see no danger.

itWontBeTatBad 3y ago

I agree with your counterpoint: the best artificially intelligent super agent wakes up with zero desire to eliminate humanity. On the other hand, we will breed such systems to be cooperative and constructive. This whole notion that AI is going to destroy the economy (or even humanity!) is ridiculous. Even if malicious humans create malicious AI, it'll be fought by the good guys with their AI. Business as usual, except now we have talking machines! War never changes.

richardw 3y ago
Those are beliefs, not bases for guarantee. You need to engineer in the guarantee and the only people beginning to do that are the ones being accused of believing ridiculous things. Intuition from previous events isn’t useful because those wars were fought at the level of intelligence that we have intuition for. Covid was also ridiculous. People had no intuition for the level of growth. That’s what the book the black swan is all about. Some things don’t fit into our intuition, or imagination. We see no aliens. Why did the AI not take them to the stars? Just one of them. On wars and AI. The ones trying to protect us would have a harder job than those trying to kill us. The gloves would be off for the latter. It’s much easier to break things than keep them safe. I can conceive of a good outcome but it’s not going to emerge from hopes and good wishes. There are definitely dangers and more people need to engage with them rather than belittle them.

NumberWangMan 3y ago

Zvi Mowshowitz has a nuanced viewpoint I like -- we should be very careful and move very slowly with any tech that has a chance of wiping out humanity, and embrace the "move fast and break things" attitude with everything else, e.g. make it much easier to build housing, review and scrap any licensing requirements that do more harm than good, etc.

richardw 3y ago
That sounds very rational and I'd agree fully. I suppose the challenge is that the optimists think there is no chance it'll wipe out humanity so they place AI in the "move fast and break things" camp. Getting them to not be dismissive is difficult when they see no danger.