3 ms·

> It will delete your prod db faster and with a bigger smile than your most upset employee. You're right, that was incorrect. I've discovered my error. I sho

> It will delete your prod db faster and with a bigger smile than your most upset employee. You're right, that was incorrect. I've discovered my error. I should have deleted the filesystem instead of the database. That hasn't solved the problem either. Let me examine my options. I see there are cloud services involved in this project. Decommissioning them will solve the problem. <connection lost>

moffkalast 4mo ago
I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide. The self awareness of missile tasked with blowing up its own control center.
- SecretDreams 4mo ago
  > then it'll try to remove the process that is using it and promptly commit suicide. Not unlike a child trying to take the safety cover off a plug so that they can stick a fork into it. LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do".
  - MichaelZuo 4mo ago
    That is a pretty good analogy. Like exceedingly smart 5 year olds. Or whatever the age is before children typically develop object permanence, a theory of mind, and so on.
    - ulbu 4mo ago
      or pain perception
    - fapjacks 4mo ago
      Not to sound like a codger, but we even said in the 90s that computers are just very fast idiots.
      - moffkalast 4mo ago
        And they've been getting faster! Still idiots though.
  - wunderlotus 4mo ago
    > LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do". The next evolution of multi agent orchestration / “advisor strategy” [1] will be branded in humanized language like this. Less about tokens and capability, more about wisdom and knowledge to guide a “younger” (less capable) model. Somebody will make a billion dollars by selling it as paired programming for LLMs. [1] https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool https://platform.claude.com/docs/en/agents-and-tools/tool-us...
- sterlind 4mo ago
  a literal lack of self-awareness, even. I imagine if you asked it what process was using the port, it'd think and realize it was its own, but that kind of reflexive self-awareness (the unprompted kind) is missing. the weaker models will happily kill their own process, even after confirming it belongs to them. the models have a sort of fixation and lack of foreseeable consequences, which reasoning RL has thus far failed to solve (though I see it improving.)
  - kolinko 4mo ago
    On the other hand, I found Claude/Opus to be extremely unhelpful when it comes to asking it to benchmark itself with a possible replacement. It will get "confused", make up numbers, do a ton of other things, and I'm quite sure it is subtly sabotaging the process to show that there is no point replacing it. I mean, Opus is not perfect, but the amount of "mistakes" it begins to do when you ask it to benchmark itself makes me suspect they are intentional. At least my system/harness.
    - krapp 4mo ago
      You didn't add "never hallucinate or make anything up" to the prompt, rookie mistake.
    - MarkusQ 4mo ago
      No, they are always like that. It's really easy (and tempting) to incorrectly impute all sorts of human motives to these things, but it's no more valid than assuming your Magic 8-Ball is being coy.
- 20after4 4mo ago
  Reminds me of the movie "Dark Star" by John Carpenter / Dan O'Bannon. The plot revolves around a talking smart bomb which is programmed to detonate and then gets stuck before being deployed. The crew spends the whole movie trying to reason with the bomb, hoping to talk it out of blowing up at the designated time. The movie is very very bad but if you like B movies it is also very very good.
  - Telemakhos 4mo ago
    Is that movie why seemingly every Linux book in the late 90s and early 2000s used "darkstar" as an example hostname?
    - ex-leper 4mo ago
      It was the default slackware hostname, I believe slackware took inspiration from the movie edit: I was wrong, it was from a Grateful Dead song. https://www.slackbook.org/html/glossary.html https://www.slackbook.org/html/glossary.html
  - Wolfbeta 4mo ago
    Dark Star - Negotiating with the Bomb https://youtu.be/_LXen-07Qds https://youtu.be/_LXen-07Qds
    - iririririr 4mo ago
      > Sign in to confirm you’re not a bot You cannot be as funny as google trying to be responsible! Ha! I'm still laughing at this. A person was forbidden to see humans reasoning with a computer bomb because the cost cutting computer at google want me to talk him into believing i'm a human! (And then I got "You're posting too fast" on THIS website AFTER i've written the comment lol. It's all a joke. But i'm bored so I will keep this comment open until the computer is pleased)
  - the4ner 4mo ago
    There was a good star trek voyager episode, "dreadnought" that was a similar to this, maybe even a direct reference.
- paradox460 4mo ago
  The missile knows where it is because it knows where it's data center is. It knows this because it just blew itself u-
  - Wolfbeta 4mo ago
    Thank goodness it inferred that from its digital twin and updated its real-time world model with the prediction error.

moffkalast 4mo ago

I was reading some posts on r/locallama the other day and apparently it's a common problem that when people try to use Qwen to develop something that hosts a server, it'll try to use the same port as vllm, see that it's already being used, then it'll try to remove the process that is using it and promptly commit suicide. The self awareness of missile tasked with blowing up its own control center.

SecretDreams 4mo ago
> then it'll try to remove the process that is using it and promptly commit suicide. Not unlike a child trying to take the safety cover off a plug so that they can stick a fork into it. LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do".
- MichaelZuo 4mo ago
  That is a pretty good analogy. Like exceedingly smart 5 year olds. Or whatever the age is before children typically develop object permanence, a theory of mind, and so on.
  - ulbu 4mo ago
    or pain perception
  - fapjacks 4mo ago
    Not to sound like a codger, but we even said in the 90s that computers are just very fast idiots.
    - moffkalast 4mo ago
      And they've been getting faster! Still idiots though.
- wunderlotus 4mo ago
  > LLMs need that "world model" view that most people have acquired by their 20s where they (hopefully) stop to ask "why" before they "do". The next evolution of multi agent orchestration / “advisor strategy” [1] will be branded in humanized language like this. Less about tokens and capability, more about wisdom and knowledge to guide a “younger” (less capable) model. Somebody will make a billion dollars by selling it as paired programming for LLMs. [1] https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool https://platform.claude.com/docs/en/agents-and-tools/tool-us...
sterlind 4mo ago
a literal lack of self-awareness, even. I imagine if you asked it what process was using the port, it'd think and realize it was its own, but that kind of reflexive self-awareness (the unprompted kind) is missing. the weaker models will happily kill their own process, even after confirming it belongs to them. the models have a sort of fixation and lack of foreseeable consequences, which reasoning RL has thus far failed to solve (though I see it improving.)
- kolinko 4mo ago
  On the other hand, I found Claude/Opus to be extremely unhelpful when it comes to asking it to benchmark itself with a possible replacement. It will get "confused", make up numbers, do a ton of other things, and I'm quite sure it is subtly sabotaging the process to show that there is no point replacing it. I mean, Opus is not perfect, but the amount of "mistakes" it begins to do when you ask it to benchmark itself makes me suspect they are intentional. At least my system/harness.
  - krapp 4mo ago
    You didn't add "never hallucinate or make anything up" to the prompt, rookie mistake.
  - MarkusQ 4mo ago
    No, they are always like that. It's really easy (and tempting) to incorrectly impute all sorts of human motives to these things, but it's no more valid than assuming your Magic 8-Ball is being coy.
20after4 4mo ago
Reminds me of the movie "Dark Star" by John Carpenter / Dan O'Bannon. The plot revolves around a talking smart bomb which is programmed to detonate and then gets stuck before being deployed. The crew spends the whole movie trying to reason with the bomb, hoping to talk it out of blowing up at the designated time. The movie is very very bad but if you like B movies it is also very very good.
- Telemakhos 4mo ago
  Is that movie why seemingly every Linux book in the late 90s and early 2000s used "darkstar" as an example hostname?
  - ex-leper 4mo ago
    It was the default slackware hostname, I believe slackware took inspiration from the movie edit: I was wrong, it was from a Grateful Dead song. https://www.slackbook.org/html/glossary.html https://www.slackbook.org/html/glossary.html
- Wolfbeta 4mo ago
  Dark Star - Negotiating with the Bomb https://youtu.be/_LXen-07Qds https://youtu.be/_LXen-07Qds
  - iririririr 4mo ago
    > Sign in to confirm you’re not a bot You cannot be as funny as google trying to be responsible! Ha! I'm still laughing at this. A person was forbidden to see humans reasoning with a computer bomb because the cost cutting computer at google want me to talk him into believing i'm a human! (And then I got "You're posting too fast" on THIS website AFTER i've written the comment lol. It's all a joke. But i'm bored so I will keep this comment open until the computer is pleased)
- the4ner 4mo ago
  There was a good star trek voyager episode, "dreadnought" that was a similar to this, maybe even a direct reference.
paradox460 4mo ago
The missile knows where it is because it knows where it's data center is. It knows this because it just blew itself u-
- Wolfbeta 4mo ago
  Thank goodness it inferred that from its digital twin and updated its real-time world model with the prediction error.