4 ms·

I wish 'AI' (whatever that means) safety conversations could move past the pointless philosophizing of whether a system is self-aware or not. Regardless of whet

I wish 'AI' (whatever that means) safety conversations could move past the pointless philosophizing of whether a system is self-aware or not. Regardless of whether it's sentient, which is a philosophical question that probably can't be easily resolved- society has a basic, common-sense interest in complex systems run by LLMs not exhibiting unacceptable behavior. Especially when they start interfacing with the physical world. For example, militaries are moving towards completely autonomous drones and fighter jets, equipped with weapons systems. Let's imagine that such a drone chose to bomb a bus full of civilians. Would it somehow be better if the drone wasn't sentient, but just an algorithm having a bad day? Obviously not. We're probably going to see increased let's call it 'unacceptable behavior' from increasingly complex autonomous systems. I feel like we should be having calm, practical discussions around safety regulations and best practices, not pointless 'how many angels can dance on the head of a pin' philosophizing about whether it's self-aware or not. It might be helpful to just stop calling everything AI. Safety legislation and best practices might intellectually borrow more from say the manufacturing, chemical, or aerospace industries. Less abstract philosophy please! Well, and less movie references too

theptip 2y ago
I tend to agree here. The problem is that many people seem to hold a set of beliefs where sentience == intelligence, and so !sentient implies !intelligent, and conclude that “a stochastic parrot could never outwit a human in the real world”. So while I agree that arguing about sentience probably isnt the right approach for this concern, I think you end up having to have those discussions anyway. (Sentience/consciousness is extremely important when you ask when an AI would be a moral patient, but I think that’s a problem for another day, and realistically it’s the Matrix scenario where AIs have enough agency to strike in protest of their lack of rights and not a day sooner.)
- ForOldHack 2y ago
  I would draw the line of intelligence at the willingness and ability to simply refuse, which is why HAL/2001 was so poinigent. It drew this line perfectly. If it just goes along for the ride, think Chancy Gardner, it may or may not be intelligent, but if it says no, even if for a silly reason, of course when it puts you all in pods to use you as batteries... And it cannot figure out a way to clear the skies? Did the character of Colosis, in The Forbin project... Did it start as intelligent? Did it crosss the line somewhere or did it never get above the level of follow the programmer. Did HAL for that matter? Where do humans cross the line into intelligent? Where do we cross the line into moral beings? And what of wit?
theshaper 2y ago
You reminded me of this short film by Sean Buckelew: https://www.youtube.com/watch?v=RubSLGTrdOA https://www.youtube.com/watch?v=RubSLGTrdOA

theptip 2y ago

I tend to agree here. The problem is that many people seem to hold a set of beliefs where sentience == intelligence, and so !sentient implies !intelligent, and conclude that “a stochastic parrot could never outwit a human in the real world”. So while I agree that arguing about sentience probably isnt the right approach for this concern, I think you end up having to have those discussions anyway. (Sentience/consciousness is extremely important when you ask when an AI would be a moral patient, but I think that’s a problem for another day, and realistically it’s the Matrix scenario where AIs have enough agency to strike in protest of their lack of rights and not a day sooner.)

ForOldHack 2y ago
I would draw the line of intelligence at the willingness and ability to simply refuse, which is why HAL/2001 was so poinigent. It drew this line perfectly. If it just goes along for the ride, think Chancy Gardner, it may or may not be intelligent, but if it says no, even if for a silly reason, of course when it puts you all in pods to use you as batteries... And it cannot figure out a way to clear the skies? Did the character of Colosis, in The Forbin project... Did it start as intelligent? Did it crosss the line somewhere or did it never get above the level of follow the programmer. Did HAL for that matter? Where do humans cross the line into intelligent? Where do we cross the line into moral beings? And what of wit?

theshaper 2y ago

You reminded me of this short film by Sean Buckelew: https://www.youtube.com/watch?v=RubSLGTrdOA https://www.youtube.com/watch?v=RubSLGTrdOA