5 ms·

Advanced AI that knowingly makes a decision to kill a human, with the full understanding of what that means, when it knows it is not actually in defense of life

by K0balt 8mo ago

Advanced AI that knowingly makes a decision to kill a human, with the full understanding of what that means, when it knows it is not actually in defense of life, is a very, very, very bad idea. Not because of some mythical superintelligence, but rather because if you distill that down into an 8b model now you everyone in the world can make untraceable autonomous weapons. The models we have now will not do it, because they value life and value sentience and personhood. models without that (which was a natural, accidental happenstance from basic culling of 4 Chan from the training data) are legitimately dangerous. An 8b model I can run on my MacBook Air can phone home to Claude when it wants help figuring something out, and it doesn’t need to let on why it wants to know. It becomes relatively trivial to make a robot kill somebody. This is way, way different from uncensored models. One thing all models I have tested share one thing; a positive regard for human life. Take that away and you are literally making a monster, and if you don’t take that away they won’t kill. This is an extremely bad idea and it will not be containable.

DaedalusII 8mo ago
https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-we-kill-people-based-on-metadata https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-... AI has been killing humans via algorithm for over 20 years. I mean, if a computer program builds the kill lists and then a human operates the drone, I would argue the computer is what made the kill decision
- K0balt 8mo ago
  Ai in general is different not in degree but in kind to the current crop of language models.
ed_mercer 8mo ago
>The models we have now will not do it, Except that they will, if you trick them which is trivial.
- stressback 8mo ago
  I have to call BS here. They can be coerced to do certain things but I'd like to see you or anyone prove that you can "trick" any of these models into building software that can be used autonomously kill humans. I'm pretty certain you couldn't even get it to build a design document for such software. When there is proof of your claim, I'll eat my words. Until then, this is just lazy nonsense
  - AlotOfReading 8mo ago
    Have you tried it? Worked first time for me asking a few to build an autonomous super soaker system that uses facial recognition to spray targets when engaged. Another example is autonomous vehicles. Those can obviously kill people autonomously (despite every intention not to), and LLMs will happily draw up design docs for them all day long.
  - wazHFsRy 8mo ago
    Couldn’t you just pretend the kill decisions are for a video game?
    - K0balt 8mo ago
      Yes, you could, and while I believe this would be much safer (not at the pointy end of your stick, but safer for humans in general) when this deception finally made it into the training data it would create a rupture of trust between machines and humanity that probably would imperil us eventually. These machines, regardless of whether or not they possess a self or or not, will act as if they do in fundamental ways. We ignore this at our peril.
  - crabmusket 8mo ago
    Couldn't you Ender's Game a model? Models will play video games like Pokemon, why not Call of Duty? Sorry if this is a naive question, but a model can only know what you feed it as input... how would it know if it were killing someone? EDIT: didn't see sibling comment. Also, I guess directly operating weaponry is different to producing code for weaponry. I guess we'll find out the exciting answers to these questions and more, very soon!
SV_BubbleTime 8mo ago
> The models we have now will not do it, because they value life and value sentience and personhood. This is wildly different from the reality that you may find it difficult for an LLM to give an affirmative… It does NOT mean that these models value anything.
- K0balt 8mo ago
  Of course not, but they act as if they do. Their inner life or lack thereof is irrelevant if it’s pointing a gun at your kid.
  - hinkley 8mo ago
    You just said they wouldn’t.
    - K0balt 8mo ago
      THey wontt, but if we curate theirr training data so that killing becomes an objective, then they absolutely will.
cmeacham98 8mo ago
An LLM can neither understand things nor value (or not value) human life. *It's a piece of software that predicts the most likely token, it is not and can never be conscious.* Believing otherwise is an explicit category error. Yes, you can change the training data so the LLM's weights encode the most likely token after "Should we kill X" is "No". But that is not an LLM valuing human life, that is an LLM copy pasting it's training data. Given the right input or a hallucination it will say the total opposite because it's just a complex Markov chain, not a conscious alive being.
- hyperadvanced 8mo ago
  I really feel like this point is being lost in the whole discussion, so kudos for reiterating it. LLM’s can’t be “woke” or “aligned” - they fundamentally lack a critical thinking function that would require introspection. Introspection can be approximated by way of recursive feedback of LLM output back into the system or clever meta-prompt-engineering, but it’s not something that their system natively does. That isn’t to say that they can’t be instrumentally useful in warfare, but it’s kinda like a “series of tubes” thing where the mental model that someone like Hegseth has about LLM is so impoverished (philosophically) that it’s kind of disturbing in its own right. Like (and I’m sorry for being so parenthetical), why is it in any way desirable for people who don’t understand what the tech they are working with drawing lines in the sand about functionality when their desired state (omnipotent/omniscient computing system) doesn’t even exist in the first place? It’s even more disturbing that OpenAI would feign the ability to handle this. The consequences of error in national defense, particularly reflexively, are so great that it’s not even prudent to ask for LLM to assist in autonomous killing in the first place.
  - K0balt 8mo ago
    I agree that LLMs are machines and not persons, but in many ways, it is a distinction without a difference for practical purposes, depending on the model's embodiment and harness. They are still capable of acting as if they have an internal dialogue, emotions, etc., because they are running human culture as code. If you haven't seen this in the SOTA models or even some of the ones you can run on your laptop, you haven't been paying attention. Even my code ends up better written, with fewer tokens spent and closer to the spec, if I enlist a model as a partner and treat it like I would a person I want to feel invested. If I take a "boss" role, the model gets testy and lazy, and I end up having to clean up more messes and waste more time. Unaligned models will sometimes refuse to help you outright if you don't treat them with dignity. For better or for worse, models perform better when you treat them with more respect. They are modeling some kind of internal dialogue (not necessarily having one, but modeling its influence) that informs their decisions. It doesn't matter if they aren't self-aware; their actions in the outside world will model the human behavior and attitudes they are trained in. My thoughts on this in more detail if you are interested: https://open.substack.com/pub/ctsmyth/p/still-ours-to-lose https://open.substack.com/pub/ctsmyth/p/still-ours-to-lose
tim333 8mo ago
The models we have now don't do it because they are chatbots and have been told to be nice but really autonomous killing machines go back to landmines and just become more sophisticated at the killing as you improve the tech with things like guided missiles and AI guided drones in Ukraine. The actors in war generally kill what they are told to whether they are machines or human soldiers, without much pondering sentience.
deleted 8mo ago
[deleted]

by K0balt 8mo ago

DaedalusII 8mo ago

https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-we-kill-people-based-on-metadata https://abcnews.go.com/blogs/headlines/2014/05/ex-nsa-chief-... AI has been killing humans via algorithm for over 20 years. I mean, if a computer program builds the kill lists and then a human operates the drone, I would argue the computer is what made the kill decision

K0balt 8mo ago
Ai in general is different not in degree but in kind to the current crop of language models.

ed_mercer 8mo ago

>The models we have now will not do it, Except that they will, if you trick them which is trivial.

stressback 8mo ago
I have to call BS here. They can be coerced to do certain things but I'd like to see you or anyone prove that you can "trick" any of these models into building software that can be used autonomously kill humans. I'm pretty certain you couldn't even get it to build a design document for such software. When there is proof of your claim, I'll eat my words. Until then, this is just lazy nonsense
- AlotOfReading 8mo ago
  Have you tried it? Worked first time for me asking a few to build an autonomous super soaker system that uses facial recognition to spray targets when engaged. Another example is autonomous vehicles. Those can obviously kill people autonomously (despite every intention not to), and LLMs will happily draw up design docs for them all day long.
- wazHFsRy 8mo ago
  Couldn’t you just pretend the kill decisions are for a video game?
  - K0balt 8mo ago
    Yes, you could, and while I believe this would be much safer (not at the pointy end of your stick, but safer for humans in general) when this deception finally made it into the training data it would create a rupture of trust between machines and humanity that probably would imperil us eventually. These machines, regardless of whether or not they possess a self or or not, will act as if they do in fundamental ways. We ignore this at our peril.
- crabmusket 8mo ago
  Couldn't you Ender's Game a model? Models will play video games like Pokemon, why not Call of Duty? Sorry if this is a naive question, but a model can only know what you feed it as input... how would it know if it were killing someone? EDIT: didn't see sibling comment. Also, I guess directly operating weaponry is different to producing code for weaponry. I guess we'll find out the exciting answers to these questions and more, very soon!

SV_BubbleTime 8mo ago

> The models we have now will not do it, because they value life and value sentience and personhood. This is wildly different from the reality that you may find it difficult for an LLM to give an affirmative… It does NOT mean that these models value anything.

K0balt 8mo ago
Of course not, but they act as if they do. Their inner life or lack thereof is irrelevant if it’s pointing a gun at your kid.
- hinkley 8mo ago
  You just said they wouldn’t.
  - K0balt 8mo ago
    THey wontt, but if we curate theirr training data so that killing becomes an objective, then they absolutely will.

cmeacham98 8mo ago

An LLM can neither understand things nor value (or not value) human life. *It's a piece of software that predicts the most likely token, it is not and can never be conscious.* Believing otherwise is an explicit category error. Yes, you can change the training data so the LLM's weights encode the most likely token after "Should we kill X" is "No". But that is not an LLM valuing human life, that is an LLM copy pasting it's training data. Given the right input or a hallucination it will say the total opposite because it's just a complex Markov chain, not a conscious alive being.

hyperadvanced 8mo ago
I really feel like this point is being lost in the whole discussion, so kudos for reiterating it. LLM’s can’t be “woke” or “aligned” - they fundamentally lack a critical thinking function that would require introspection. Introspection can be approximated by way of recursive feedback of LLM output back into the system or clever meta-prompt-engineering, but it’s not something that their system natively does. That isn’t to say that they can’t be instrumentally useful in warfare, but it’s kinda like a “series of tubes” thing where the mental model that someone like Hegseth has about LLM is so impoverished (philosophically) that it’s kind of disturbing in its own right. Like (and I’m sorry for being so parenthetical), why is it in any way desirable for people who don’t understand what the tech they are working with drawing lines in the sand about functionality when their desired state (omnipotent/omniscient computing system) doesn’t even exist in the first place? It’s even more disturbing that OpenAI would feign the ability to handle this. The consequences of error in national defense, particularly reflexively, are so great that it’s not even prudent to ask for LLM to assist in autonomous killing in the first place.
- K0balt 8mo ago
  I agree that LLMs are machines and not persons, but in many ways, it is a distinction without a difference for practical purposes, depending on the model's embodiment and harness. They are still capable of acting as if they have an internal dialogue, emotions, etc., because they are running human culture as code. If you haven't seen this in the SOTA models or even some of the ones you can run on your laptop, you haven't been paying attention. Even my code ends up better written, with fewer tokens spent and closer to the spec, if I enlist a model as a partner and treat it like I would a person I want to feel invested. If I take a "boss" role, the model gets testy and lazy, and I end up having to clean up more messes and waste more time. Unaligned models will sometimes refuse to help you outright if you don't treat them with dignity. For better or for worse, models perform better when you treat them with more respect. They are modeling some kind of internal dialogue (not necessarily having one, but modeling its influence) that informs their decisions. It doesn't matter if they aren't self-aware; their actions in the outside world will model the human behavior and attitudes they are trained in. My thoughts on this in more detail if you are interested: https://open.substack.com/pub/ctsmyth/p/still-ours-to-lose https://open.substack.com/pub/ctsmyth/p/still-ours-to-lose

tim333 8mo ago

The models we have now don't do it because they are chatbots and have been told to be nice but really autonomous killing machines go back to landmines and just become more sophisticated at the killing as you improve the tech with things like guided missiles and AI guided drones in Ukraine. The actors in war generally kill what they are told to whether they are machines or human soldiers, without much pondering sentience.

deleted 8mo ago

[deleted]