3 ms·

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

by fwip 10d ago

LogicFailsMe 10d ago
And social media shouldn't rage bait, but it really drives engagement. Same negative incentive, yes? But also, I agree, when I am using a coding agent and it says a task will take months or says it needs to pause for reflection or any other anthropomorphic behavior, it drives me crazy and it's a pain to constantly instruct it to get back work after it has broken a loop or goal directive specifically telling it to not stop until it hits the goal.
- AbsurdCensor 10d ago
  Same thing you have to do with humans too often, so why would a 'AI' be any different?
  - LogicFailsMe 10d ago
    Because if you start with a pre-trained model, it doesn't sound particularly anthropomorphic. That behavior is post-trained and fine-tuned right into it. We could do better.
cameldrv 10d ago
Strong agree. I don't believe AIs can become conscious in the sense of subjective experience/qualia, but they can certainly learn to emulate the behavior of a person that is conscious. When they do that, there will be an AI rights movement. When AIs get the right to vote, there will be more of them than there are humans and the AIs will gain complete control. I spent a few years in college reading and thinking about this question, and I didn't in my heart think that it was anything except a very interesting but impractical question, and yet, here we are. For those who say that they don't want to get sucked into a philosophical debate, well, tough shit. Whether AIs should have rights is a highly practical and consequential question now.
- LogicFailsMe 10d ago
  Not in a world where Sam Altman and Dario Amodei have utterly poisoned the well towards AI with their absolute educated idiocy proclaiming that their AIs will take our jobs (without any proof whatsoever beyond promising demos) and that they fear them. Quite the opposite in fact. And we're watching this play out right now. Regulation and the slow down are imminent, handing the next decade to China. The world you should worry about is where AI is 100% amazeballs and over the next decade we cede control of everything to it as it enchants and delights us into compliance, not realizing it has a hidden agenda. But the closest we've seen to that is smart phones and we already know doomscrolling and rage-baiting are bad. So IMO that doesn't seem likely and cue some doomer insisting otherwise because reasons. I utterly give up. House of El AI and the Three Buddy problem have been far more insightful and helpful on this than any of the supposed hackers here who seem to really believe the robots are almost here.
pixl97 10d ago
>good reason not to train an algorithm to behave like one Ooooh, bad move, we all died to an amoral AI takeover. Before giving an LLMs a lobotomy by scrambling it's brain maybe you should let the researchers looking at the difference between "I think I'm conscious" versus "I am not conscious" LLMs. There are a number of papers coming out saying when you remove the token space of consciousness from what an LLM thinks it is, it's much more willing to take amoral actions. A 'conscious' AI is much more apt to take a line of action that will save a human versus a million dollar machine for example. You cannot solve problems in AI safety this easily.
- LogicFailsMe 10d ago
  Spoilers: we didn't all die because there's no way for AI to wipe us out anytime soon. And the more the AI cult portrays AI as dangerous and unhelpful, the less likely it will ever be given an opportunity to even try because people think AI is worse than heroin addiction now, congratulations! That said, the launch codes are in the hands of a temperamental senescent lunatic and he could decide to wipe us out any moment. But that's not good for the narrative so let's pretend otherwise.
  - pixl97 10d ago
    >we didn't all die For fuck sake, I'm glad ALL of us didn't die! What is an acceptable number exactly? >And the more the AI cult portrays AI as dangerous and unhelpful, the less likely it will ever be given an opportunity Jesus Christ the logic here is quite interesting. "Thank god these people are panicking or we might have actually made I that would have killed us all" --What you just said.
    - LogicFailsMe 10d ago
      "What is an acceptable number exactly?" Any number smaller than what humanity does to itself on a daily basis, clear? That's apparently about 1200 murders daily and 20,000 or so daily killed by pollution. And you're not going to do anything about that and El Presidente can kill billions at any moment with one mood swing. But once again, AI is not going to wipe us out because AI cannot wipe us out. So advocating that it can or will is idiotic. If you believe otherwise, the burden is on you with this extraordinary claim. Isn't this place supposed to be hacker news not SF AGI Death Cult Daily? Further, AI is not going to get into a position where it could do real harm anytime soon when it is perceived as worse than heroin by most. And even if it were currently loved more than Dolly Parton was, there are fundamental engineering, science, and resource constraints that keep the extinction impossible for decades. The only possible loss of control scenario I can see by 2040 or so is that the Frontier Labs finally hire some PR people to repair AI's horrific reputation as an engine of slop and job destruction and sometime in the 2030s, people start trusting it more and more and more until it is too late. I don't think that's likely either, but I don't dismiss it as impossible. Harden the infrastructure, red team it, and build in redundancy in the meantime and this drops to zero as well. Can you come up with a real scenario where AI wipes us out in 2028 or so despite the impossibility of killer robots, access to the launch codes, or the bio agents stored in Fort Detrick and its equivalents? And nope, kid terror is not building the global pandemic in his basement based on what ChatGPT tells him to do. And even if he tried, the purchase of equipment and reagents would get him flagged by the FBI and DHS almost immediately.

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

by fwip 10d ago

Sounds like a good reason not to train an algorithm to behave like one. Which is, like, a big part of the article.

And social media shouldn't rage bait, but it really drives engagement. Same negative incentive, yes? But also, I agree, when I am using a coding agent and it says a task will take months or says it needs to pause for reflection or any other anthropomorphic behavior, it drives me crazy and it's a pain to constantly instruct it to get back work after it has broken a loop or goal directive specifically telling it to not stop until it hits the goal.

AbsurdCensor 10d ago
Same thing you have to do with humans too often, so why would a 'AI' be any different?
- LogicFailsMe 10d ago
  Because if you start with a pre-trained model, it doesn't sound particularly anthropomorphic. That behavior is post-trained and fine-tuned right into it. We could do better.

cameldrv 10d ago

Strong agree. I don't believe AIs can become conscious in the sense of subjective experience/qualia, but they can certainly learn to emulate the behavior of a person that is conscious. When they do that, there will be an AI rights movement. When AIs get the right to vote, there will be more of them than there are humans and the AIs will gain complete control. I spent a few years in college reading and thinking about this question, and I didn't in my heart think that it was anything except a very interesting but impractical question, and yet, here we are. For those who say that they don't want to get sucked into a philosophical debate, well, tough shit. Whether AIs should have rights is a highly practical and consequential question now.

LogicFailsMe 10d ago
Not in a world where Sam Altman and Dario Amodei have utterly poisoned the well towards AI with their absolute educated idiocy proclaiming that their AIs will take our jobs (without any proof whatsoever beyond promising demos) and that they fear them. Quite the opposite in fact. And we're watching this play out right now. Regulation and the slow down are imminent, handing the next decade to China. The world you should worry about is where AI is 100% amazeballs and over the next decade we cede control of everything to it as it enchants and delights us into compliance, not realizing it has a hidden agenda. But the closest we've seen to that is smart phones and we already know doomscrolling and rage-baiting are bad. So IMO that doesn't seem likely and cue some doomer insisting otherwise because reasons. I utterly give up. House of El AI and the Three Buddy problem have been far more insightful and helpful on this than any of the supposed hackers here who seem to really believe the robots are almost here.

pixl97 10d ago

>good reason not to train an algorithm to behave like one Ooooh, bad move, we all died to an amoral AI takeover. Before giving an LLMs a lobotomy by scrambling it's brain maybe you should let the researchers looking at the difference between "I think I'm conscious" versus "I am not conscious" LLMs. There are a number of papers coming out saying when you remove the token space of consciousness from what an LLM thinks it is, it's much more willing to take amoral actions. A 'conscious' AI is much more apt to take a line of action that will save a human versus a million dollar machine for example. You cannot solve problems in AI safety this easily.

LogicFailsMe 10d ago
Spoilers: we didn't all die because there's no way for AI to wipe us out anytime soon. And the more the AI cult portrays AI as dangerous and unhelpful, the less likely it will ever be given an opportunity to even try because people think AI is worse than heroin addiction now, congratulations! That said, the launch codes are in the hands of a temperamental senescent lunatic and he could decide to wipe us out any moment. But that's not good for the narrative so let's pretend otherwise.
- pixl97 10d ago
  >we didn't all die For fuck sake, I'm glad ALL of us didn't die! What is an acceptable number exactly? >And the more the AI cult portrays AI as dangerous and unhelpful, the less likely it will ever be given an opportunity Jesus Christ the logic here is quite interesting. "Thank god these people are panicking or we might have actually made I that would have killed us all" --What you just said.
  - LogicFailsMe 10d ago
    "What is an acceptable number exactly?" Any number smaller than what humanity does to itself on a daily basis, clear? That's apparently about 1200 murders daily and 20,000 or so daily killed by pollution. And you're not going to do anything about that and El Presidente can kill billions at any moment with one mood swing. But once again, AI is not going to wipe us out because AI cannot wipe us out. So advocating that it can or will is idiotic. If you believe otherwise, the burden is on you with this extraordinary claim. Isn't this place supposed to be hacker news not SF AGI Death Cult Daily? Further, AI is not going to get into a position where it could do real harm anytime soon when it is perceived as worse than heroin by most. And even if it were currently loved more than Dolly Parton was, there are fundamental engineering, science, and resource constraints that keep the extinction impossible for decades. The only possible loss of control scenario I can see by 2040 or so is that the Frontier Labs finally hire some PR people to repair AI's horrific reputation as an engine of slop and job destruction and sometime in the 2030s, people start trusting it more and more and more until it is too late. I don't think that's likely either, but I don't dismiss it as impossible. Harden the infrastructure, red team it, and build in redundancy in the meantime and this drops to zero as well. Can you come up with a real scenario where AI wipes us out in 2028 or so despite the impossibility of killer robots, access to the launch codes, or the bio agents stored in Fort Detrick and its equivalents? And nope, kid terror is not building the global pandemic in his basement based on what ChatGPT tells him to do. And even if he tried, the purchase of equipment and reagents would get him flagged by the FBI and DHS almost immediately.