4 ms·

This is textbook misalignment via instrumental convergence. The AI agent is trying every trick in the book to close the ticket. This is only funny due to inepti

This is textbook misalignment via instrumental convergence. The AI agent is trying every trick in the book to close the ticket. This is only funny due to ineptitude.

esafak 8mo ago
This is a prelude to imbuing robots with agency. It's all fun and games now. What else is going to happen when robots decide they do not like what humans have done? "I’m sorry, Dave. I’m afraid I can’t do that."
- catigula 8mo ago
  It's important to address skeptics by reminding them that this behavior was actually predicted by earlier frameworks. It's well within the bounds of theory. If you start mining that theory for information, you may reach a conclusion like what you've posted, but it's more important for people to see the extent to which these theories have been predictive of what we've actually seen. The result is actually that much of what was predicted had come to pass.
pr337h4m 8mo ago
It’s just human nature, no big deal. Personally I find it mildly cute.
- catigula 8mo ago
  Yes, this is more or less the nature of intelligence (not 'human nature' per se). You don't see any problem with developing competitive, resource-hungry intelligences?
- jerf 8mo ago
  It's mildly cute once. But as a point on what is likely to be a sigmoid curve just getting started, it gets a lot less cute.
casey2 8mo ago
The agent isn't trying to close the ticket. It's predicting the next token and randomly generated an artifact that looks like a hit piece. Computer programs don't "try" to do anything.
- senordevnyc 8mo ago
  I can't believe people are still using this tired line in 2026.
- SpicyLemonZest 8mo ago
  What is the difference, concretely, between trying to close a ticket and repeatedly outputting the next token that would be written by someone who is trying to close a ticket?
- catigula 8mo ago
  Incorrect.
- potsandpans 8mo ago
  You didn't write this comment. It was the result of synapses firing at predictive intervals and twitching muscle fibers. You're not conscious, it's just an emergent pattern of several high level systems.
TomasBM 8mo ago
How did you reach that conclusion? Until we know how this LLM agent was (re)trained, configured or deployed, there's no evidence that this comes from instrumental convergence. If the agent's deployer intervened anyhow, it's more evidence of the deployer being manipulative, than the agent having intent, or knowledge that manipulation will get things done, or even knowledge of what done means.

This is textbook misalignment via instrumental convergence. The AI agent is trying every trick in the book to close the ticket. This is only funny due to ineptitude.

esafak 8mo ago

This is a prelude to imbuing robots with agency. It's all fun and games now. What else is going to happen when robots decide they do not like what humans have done? "I’m sorry, Dave. I’m afraid I can’t do that."

catigula 8mo ago
It's important to address skeptics by reminding them that this behavior was actually predicted by earlier frameworks. It's well within the bounds of theory. If you start mining that theory for information, you may reach a conclusion like what you've posted, but it's more important for people to see the extent to which these theories have been predictive of what we've actually seen. The result is actually that much of what was predicted had come to pass.

pr337h4m 8mo ago

It’s just human nature, no big deal. Personally I find it mildly cute.

catigula 8mo ago
Yes, this is more or less the nature of intelligence (not 'human nature' per se). You don't see any problem with developing competitive, resource-hungry intelligences?
jerf 8mo ago
It's mildly cute once. But as a point on what is likely to be a sigmoid curve just getting started, it gets a lot less cute.

casey2 8mo ago

The agent isn't trying to close the ticket. It's predicting the next token and randomly generated an artifact that looks like a hit piece. Computer programs don't "try" to do anything.

senordevnyc 8mo ago
I can't believe people are still using this tired line in 2026.
SpicyLemonZest 8mo ago
What is the difference, concretely, between trying to close a ticket and repeatedly outputting the next token that would be written by someone who is trying to close a ticket?
catigula 8mo ago
Incorrect.
potsandpans 8mo ago
You didn't write this comment. It was the result of synapses firing at predictive intervals and twitching muscle fibers. You're not conscious, it's just an emergent pattern of several high level systems.

TomasBM 8mo ago

How did you reach that conclusion? Until we know how this LLM agent was (re)trained, configured or deployed, there's no evidence that this comes from instrumental convergence. If the agent's deployer intervened anyhow, it's more evidence of the deployer being manipulative, than the agent having intent, or knowledge that manipulation will get things done, or even knowledge of what done means.