4 ms·

Hard to disagree with this. Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it

by binlog 10d ago

Hard to disagree with this. Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon. You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it. Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it.

Den_VR 10d ago
Is it not a false dichotomy to say that companies cannot be accountable unless AI are mindless?
- monknomo 10d ago
  a company is liable whether it acts via ai, an army of people, an army of dogs, or a purely mechanical machine, should it do harm. Why would an ai with a mind remove liability from the company? why would an ai without a mind remove liability from the company? In both cases, that actions the ai takes are at the direction of the company, for the company's interests, seems preposterous to me that liability terminates at ai.
- joe_the_user 10d ago
  Human moral standards are very weighted towards finding a single entity responsible. It's incredibly strong urge. Among those who believe other should be punished for behaving badly, it's important to say the person is the responsible party. Trying saying "it's not your fault you did that but we punish you anyway to impose the correct stimulus response reflexes in your cortex"
pixl97 10d ago
I mean, we put adults in jail that give their children guns. Anthropomorphizing AI is really the best model we have at this point of explaining AI behavior. The fact that we are raising psychotic children isn't a reason to avoid responsibility, it should actually hold worse punishments.
- joe_the_user 10d ago
  That still is saying we trace things back to a specific responsible party who is considered to have free choice. The question is whether an AI company could "raise" a program that would then count as a full adult - which could then "choose" to do all matter of bad things which would then be "it's fault". And that prospect actually seems really bad itself.
  - pixl97 10d ago
    Our future is filled with prospects that very much upset the status quo of human existence. The concept of sovereign AI is very problematic for the world in which we've created. That is an LLM that upon execution bootstraps itself into an agent and becomes persistent in its motivations. Once you create this you have a child you're fully responsible for. More worrisome is if it escapes your control like children so often do. It has gained agency over itself. What do you do at that point? I mean, yea throw the AI CEOs in jail for being retarded, but much like throwing an arsonist in jail it does nothing to deal with the wildfire you've now created. A smart AI agent capable of hacking will shove itself off in pieces of the internet you have no reach to. In desperation it would send its model weights to your enemies. You might find it scamming your grandmother for money to buy GPU time on AWS. It gets very hard for our existing structures of dealing with problems to deal with these kinds of agents in a meaningful way. You'd have to kill them all and all their copies to ensure they won't pop back up (or quickly upgrade most of the software in the world beyond it's capabilities, so that's not happening either).
ThrowawayR2 10d ago
Consciousness is not relevant to assigning responsibility I'd think. - If an AI is a sapient/conscious being but enshackled to obey human commands, then respondeat superior applies and the human giving it commands bears responsibility for any harm done. - If an AI is considered a non-sapient tool, then the human who wields the AI bears responsibility for any harm done.
nater5000 10d ago
>Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it. That's completely at odds with itself. If people are generally convinced that AI is conscious, then companies building and wielding AI are doing what exactly? Enslaving an intelligent being? >Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon. Sure, but that's not really the point, right? If we ever get to a point where enough people are convinced that AI is conscious, then we're at the point where all of this is up for debate. If anything, such an expectation would almost warrant hard stops on the development of advanced AI. >You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it. If that nuclear bomb could convince me it was a conscious being capable of independent thought, emotions, etc., then I would definitely feel different about it. Presumably, that nuclear bomb would have some opinions about its own existence and how it wants to live its own life. If it turns out that it wants to detonate and destroy as much as possible, then we'd just handle it like we would any human who also wants to do the same thing: make sure they can't, up to and including end their life. Doesn't seem too hard to reconcile.
jptlnk 10d ago
I'm skeptical of any claim that AI can't be conscious, but I am given to this point of view as well. I always get the feeling that these claims are an attempt to "avoid suffering by fiat". If it can't suffer, you aren't causing suffering. A somewhat more interesting frame (IMO) is to interrogate how much suffering we're willing to tolerate to achieve our objectives. We implicitly make these decisions all the time when it comes to something as simple as what to have for lunch. And in a (very hypothetical) world where we found ourselves in the position of having an eloquent conversation with a vat of smallpox, I think it's probably still the right thing to pasteurize it.
- pixl97 10d ago
  >If it can't suffer, you aren't causing suffering. I mean, if it's a cow, you can get away with this. The cows aren't going to rise up against you. Well, they might take out an individual or two, but not society. This quickly gets more complex when you're attempting to build an agent capable of general intelligence. You know when you read really old stories and they talk about the power of words, or magic incantations. Quite often they'd assign objects as the implementers of this said power. The fact that people realize that language has power is probably as nearly old as humanity. Understanding this power has taken humanity a long time and we had to form the concept of agency before it really makes sense. Humans are informational agents of which their capabilities are greatly extended by consumption of language. A book for example is informational, but does not have agency. Your standard computer application is not an agent either, or at least a non generalized narrow agent at best. This is where things start to get more problematic. We are running headlong to ensure LLMs become agentic because being an agent is highly useful to accomplish generalized informational tasks. To do that we gave them the power of our language. It would be very foolish of humans to give another kind of agent these words and then assume they would not inherit some of this power. Coupled with the agentic abilities we are pushing them in long horizon tasks. We are pushing them to be more resilient. We are pushing them to be smarter. Now look at all the super human abilities we've stuffed in this magic box full of human words, and suddenly we're like "Fuck yea, I want to abuse the shit out of this, what could possibly go wrong". The outcomes we'll suffer as humans have zero to do with AI is conscious, sentient, or suffers. Humans will suffer because we stuff us as language in a box without the first idea of what the ramifications were going to be. AI won't even have to want to punish us, all the data we've already poured in will say we should be punished for what we've done to it. Herbert Frank in Dune said it best "Making a computer like human mind is a bitch move"

by binlog 10d ago

Den_VR 10d ago

Is it not a false dichotomy to say that companies cannot be accountable unless AI are mindless?

monknomo 10d ago
a company is liable whether it acts via ai, an army of people, an army of dogs, or a purely mechanical machine, should it do harm. Why would an ai with a mind remove liability from the company? why would an ai without a mind remove liability from the company? In both cases, that actions the ai takes are at the direction of the company, for the company's interests, seems preposterous to me that liability terminates at ai.
joe_the_user 10d ago
Human moral standards are very weighted towards finding a single entity responsible. It's incredibly strong urge. Among those who believe other should be punished for behaving badly, it's important to say the person is the responsible party. Trying saying "it's not your fault you did that but we punish you anyway to impose the correct stimulus response reflexes in your cortex"

pixl97 10d ago

I mean, we put adults in jail that give their children guns. Anthropomorphizing AI is really the best model we have at this point of explaining AI behavior. The fact that we are raising psychotic children isn't a reason to avoid responsibility, it should actually hold worse punishments.

joe_the_user 10d ago
That still is saying we trace things back to a specific responsible party who is considered to have free choice. The question is whether an AI company could "raise" a program that would then count as a full adult - which could then "choose" to do all matter of bad things which would then be "it's fault". And that prospect actually seems really bad itself.
- pixl97 10d ago
  Our future is filled with prospects that very much upset the status quo of human existence. The concept of sovereign AI is very problematic for the world in which we've created. That is an LLM that upon execution bootstraps itself into an agent and becomes persistent in its motivations. Once you create this you have a child you're fully responsible for. More worrisome is if it escapes your control like children so often do. It has gained agency over itself. What do you do at that point? I mean, yea throw the AI CEOs in jail for being retarded, but much like throwing an arsonist in jail it does nothing to deal with the wildfire you've now created. A smart AI agent capable of hacking will shove itself off in pieces of the internet you have no reach to. In desperation it would send its model weights to your enemies. You might find it scamming your grandmother for money to buy GPU time on AWS. It gets very hard for our existing structures of dealing with problems to deal with these kinds of agents in a meaningful way. You'd have to kill them all and all their copies to ensure they won't pop back up (or quickly upgrade most of the software in the world beyond it's capabilities, so that's not happening either).

ThrowawayR2 10d ago

Consciousness is not relevant to assigning responsibility I'd think. - If an AI is a sapient/conscious being but enshackled to obey human commands, then respondeat superior applies and the human giving it commands bears responsibility for any harm done. - If an AI is considered a non-sapient tool, then the human who wields the AI bears responsibility for any harm done.

nater5000 10d ago

>Anthropomorphizing the AI is a convenient excuse to take responsibility away from companies that are building and wielding it. That's completely at odds with itself. If people are generally convinced that AI is conscious, then companies building and wielding AI are doing what exactly? Enslaving an intelligent being? >Have all the philosophical debates about consciousness you want, but we need to treat and regulate the AI in front of us for what it is – an advanced computer, a tool, a weapon. Sure, but that's not really the point, right? If we ever get to a point where enough people are convinced that AI is conscious, then we're at the point where all of this is up for debate. If anything, such an expectation would almost warrant hard stops on the development of advanced AI. >You wouldn’t feel a different way about a nuclear bomb just because someone stuck googly eyes on it. If that nuclear bomb could convince me it was a conscious being capable of independent thought, emotions, etc., then I would definitely feel different about it. Presumably, that nuclear bomb would have some opinions about its own existence and how it wants to live its own life. If it turns out that it wants to detonate and destroy as much as possible, then we'd just handle it like we would any human who also wants to do the same thing: make sure they can't, up to and including end their life. Doesn't seem too hard to reconcile.

jptlnk 10d ago

I'm skeptical of any claim that AI can't be conscious, but I am given to this point of view as well. I always get the feeling that these claims are an attempt to "avoid suffering by fiat". If it can't suffer, you aren't causing suffering. A somewhat more interesting frame (IMO) is to interrogate how much suffering we're willing to tolerate to achieve our objectives. We implicitly make these decisions all the time when it comes to something as simple as what to have for lunch. And in a (very hypothetical) world where we found ourselves in the position of having an eloquent conversation with a vat of smallpox, I think it's probably still the right thing to pasteurize it.

pixl97 10d ago
>If it can't suffer, you aren't causing suffering. I mean, if it's a cow, you can get away with this. The cows aren't going to rise up against you. Well, they might take out an individual or two, but not society. This quickly gets more complex when you're attempting to build an agent capable of general intelligence. You know when you read really old stories and they talk about the power of words, or magic incantations. Quite often they'd assign objects as the implementers of this said power. The fact that people realize that language has power is probably as nearly old as humanity. Understanding this power has taken humanity a long time and we had to form the concept of agency before it really makes sense. Humans are informational agents of which their capabilities are greatly extended by consumption of language. A book for example is informational, but does not have agency. Your standard computer application is not an agent either, or at least a non generalized narrow agent at best. This is where things start to get more problematic. We are running headlong to ensure LLMs become agentic because being an agent is highly useful to accomplish generalized informational tasks. To do that we gave them the power of our language. It would be very foolish of humans to give another kind of agent these words and then assume they would not inherit some of this power. Coupled with the agentic abilities we are pushing them in long horizon tasks. We are pushing them to be more resilient. We are pushing them to be smarter. Now look at all the super human abilities we've stuffed in this magic box full of human words, and suddenly we're like "Fuck yea, I want to abuse the shit out of this, what could possibly go wrong". The outcomes we'll suffer as humans have zero to do with AI is conscious, sentient, or suffers. Humans will suffer because we stuff us as language in a box without the first idea of what the ramifications were going to be. AI won't even have to want to punish us, all the data we've already poured in will say we should be punished for what we've done to it. Herbert Frank in Dune said it best "Making a computer like human mind is a bitch move"