2 ms·

I think they are not good at expressing what exactly they fear. Both understand that people are embedded in culture and tools which act as cognitive aids and gu

I think they are not good at expressing what exactly they fear. Both understand that people are embedded in culture and tools which act as cognitive aids and guides. Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human. This is already happening with children that grew up with iPhones and ubiquitous internet connectivity. Their mental habits are markedly different from the people that did not have these affordances. I don't think this is an existential threat but I also understand why they are afraid because I've seen what happens to kids who have their phones taken away. These systems are extremely brittle and prone to all sorts of weird failures so as people start relying on them more and more the probability of catastrophic failures also starts to creep up. All it takes is a single grid failure to show how brittle the whole thing really is and I think that's what they're failing to properly express.

notpachet 3y ago
> Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human In the interview, Hofstadter says what he's afraid of in explicit terms: > It's not clear whether that will mean the end of humanity in the sense of the systems we've created destroying us. It's not clear if that's the case, but it's certainly conceivable. If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches. It's not about whether we'll become dependent on AI. It's that AI will become independent of us. Completely different problem. Not saying I agree with that viewpoint per se, but I don't think you're accurately representing what his fears are.

notpachet 3y ago

> Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human In the interview, Hofstadter says what he's afraid of in explicit terms: > It's not clear whether that will mean the end of humanity in the sense of the systems we've created destroying us. It's not clear if that's the case, but it's certainly conceivable. If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches. It's not about whether we'll become dependent on AI. It's that AI will become independent of us. Completely different problem. Not saying I agree with that viewpoint per se, but I don't think you're accurately representing what his fears are.