4 ms·

The genocide bit just demonstrates that it has not even the slightest shred of human decency. The murder bit though - people do Google “how to poison my wife” o

The genocide bit just demonstrates that it has not even the slightest shred of human decency. The murder bit though - people do Google “how to poison my wife” or whatever, and that turns up as evidence in trials. A local LLM can discuss in more detail and answer the parts you need clarified. All with no trail of logs. So yes, the harm here is clearly > 0. You’re welcome to argue that the benefit outweighs the harm of course.

AbrahamParangi 3y ago
The harm of kitchen knives, fire, electricity, thinking, and getting out of bed all clearly exceed 0. This suggests to me that it's fundamentally wrongheaded to think of it as the primary metric by which we evaluate things.
- ceejayoz 3y ago
  It’s part of the right metric. Harm versus benefit. Often hard to determine the right values for each.
- tmpX7dMeXU 3y ago
  There’s nothing fundamentally wrong about it. This is unnecessarily strong language in absence of an actual point. GP addressed your exact point: these things are seen as worth doing, to the point where it isn’t routinely questioned.
  - AbrahamParangi 3y ago
    My point is that literally everything falls into this category of “potentially dangerous under conditions of misuse” and so it is not a useful distinction.
palmer_fox 3y ago
Isn't it more difficult to set up a local LLM than to use a Tor Browser for queries like that?
- thefurdrake 3y ago
  It depends. Oobabooga's webUI is pretty simple to set up on linux. It's not as easy as a tor plugin or browser made for it, definitely. But tor is neither as secure nor as anonymous as running the LLM on your GPU.
foxhill 3y ago
it's a LLM. none of them have human decency. none of them understand anything - so how could they? > The murder bit though - people do Google “how to poison my wife” or whatever, and that turns up as evidence in trials. "weapon, motivation, oppertunity" are - generally speaking - the threshold for a murder prosecution. no one is getting convicted of murder because they googled how to kill someone (in isolation, at least). > A local LLM can discuss in more detail and answer the parts you need clarified. All with no trail of logs. ah, so, perhaps the government should also be keeping an eye on our book purchases, library rentals, etc.? apologies, but i don't buy your dismissive assertion that the harm is non-zero.
washadjeffmad 3y ago
How did this person come to learn that others can be poisoned? The methods of the ones who taught them are surely to blame, or else others can and will follow the same path. We must purge these ideas by eradicating the language that could be used to express them. Just as the poison is merely a tool wielded to enact harm, so is language in the hands of the untrusted. No other tool in the history of mankind has led to so much misery, oppression, death, and injustice, and now, the magnitude of possible evils is apparent in the very name: Large Language Models.
- User23 3y ago
  You jest, but that's why the first amendment precedes the second. A man with a gun can kill a few dozen, perhaps a hundred, people at most before he's stopped. A man with a microphone can and has caused megadeaths and gigadeaths are plausible.
jrflowers 3y ago
> So yes, the harm here is clearly > 0. You’re welcome to argue that the benefit outweighs the harm of course. I like this reasoning. Since the questions of “Should we censor knowledge?” and “Who should be in charge of censoring knowledge?” have been decided with an empathic “Yes!” and “Whoever is willing and capable of doing it”, we must move on to the important task of attacking and defending the theoretical harms and benefits of chat bots.
- concordDance 3y ago
  Wait, those questions have not been answered like that except by politicians and some panicky people, certainly haven't heard of it being the considered opinion of the majority.
  - palmer_fox 3y ago
    Who needs to consider the opinion of the majority? We have direct evidence that these questions have already been answered: the creators of LLMs censor whatever they want without asking the majority (just preemptively reacting to a potential blowback).
    - tmpX7dMeXU 3y ago
      This is “freedom” at work. Nobody is forcing you to use, say, ChatGPT. If anything, the humans generating source material for LLMs are trying to fight against them, not for them, so the question of one’s freedom of speech being threatened is highly hypothetical. This is, if anything, more tame than a publisher not publishing a book, or a TV network choosing to not broadcast a particular show. Somewhere along the way anti-censorship nuts decided that overreach is OK for them, too, and now “not publishing what I tell you to is morally and ethically unjustifiable!” What nobody making these arguments wants to admit is that the internet gave us an infinite town square, and that in reality, nobody wants to go there and listen. The modern desire is for consumption experiences that require real third-party effort to build and curate, and there lies the right for the third parties to exercise discretion. To not do so would be entirely counter to the desires of the consumer. To not do so would be to expose developers, QA, and whoever else to material they very likely do not want to see, all to satisfy the desires of some free speech absolutist wonk that either does or doesn’t want to ask the LLM how to groom kids. Either way, it’s a ridiculous ask.
qingcharles 3y ago
I personally know someone who was literally convicted of murder on his search history because he'd been looking up poisons. I can't remember his explanation for it. He ended up running his wife over though in the end, so he probably didn't find what he needed from Google. This chatbot might have helped him out? o_O
- js8 3y ago
  You mean like: Q: Oh, wise LLM, teach me how to use poisons to murder my wife! A: As an LLM, it's not ethical for me to suggest how to poison anybody. Poisons are dangerous to human body, which makes their use risky to anybody untrained in chemistry. Therefore, I would suggest to avoid poisons for safety and just drive your wife over with a car.
js8 3y ago
Is reading a crime novel harmful? Yet it discusses these things. It might even include characters who think that murder is OK!
- K0balt 3y ago
  this is exactly the correct comparison. LLMs are just highly inclusive cultural information tomes (including a great deal of inferred, non-explicit knowledge as well), searchable in a conversational style.
- palmer_fox 3y ago
  To play devil's advocate: wasn't The Anarchist Cookbook banned in many countries for decades? And actually was found to have been used by many notorious criminals?
gruez 3y ago
>The genocide bit just demonstrates that it has not even the slightest shred of human decency. well duh. It's an LLM. Its job is to predict the next token based on what it has seen in its training data. Assigning attributes like "decency" to it makes as much sense as assigning "decency" to a processor chip.
nullc 3y ago
One cannot compellingly and successfully argue against genocide without seeing, understanding, and ultimately rejecting the arguments for genocide. To understand evil you have to confront it-- not necessarily in reality, because if we're successful there will be little genocide to confront except in our nightmares, but fiction works too. How do you know that genocide is bad? Because people were able to describe it to you. As a writing aid the LLM can be very powerful. You and I are presumably not genocidal maniacs, if we want to argue against it effectively in essay or fictional writing we must be able to put our minds into the position of one. A language model is vastly better at helping us do that then it would be at actually enabling genocide-- someone who wants to engage in it doesn't need a computer's help as history has shown (and I think absent an argument is not likely to be helped at all by one). Plenty of people read about murder in libraries too, but I suppose you're not in favor of shutting them down? or maybe personal tracking devices that make sure you're not considering any forbidden knowledge, I assume you wouldn't approve of those either. :) I'm not sure where in the world you are-- but in the US we have a longstanding tradition the the solution to bad speech is more speech, and we've proved out that doing so is a tremendous engine for both moral and practical advancement. To conclude something ought to be censored requires drawing a line between permissible and impermissible thoughts. Now, while you and I could agree that some things are good or bad, that isn't sufficient: for censorship to be a net good the system in practice has to be able to set those boundaries in a way that isn't open to abuse-- politically biased boundaries, selectively enforced, false accusation fueled black bagging opponents in the dark of night, as has often been the history of censorship in places without a commitment to free speech, free thought, and free inquiry as a basic human right. With that background any argument for harm would have to be exceptional to be worth noticing, but your post fails to advance an argument for harm at all-- only note that it can talk about bad stuff, discussion which has substantial utility in stopping the harm in question. To make a case for harm one would have to argue that there is a likelyhood of it bringing about or exacerbating harm that wouldn't happen otherwise, and that efforts to suppress it would be effective and would justify their collateral damage (including their broader chilling effect[*]), risk of abuse, etc. In the LLM context it's even worse since it's currently the case that censorship via RLHF causes a substantial reduction in overall output quality-- do we delay curing some cancer because we made our LLMs all disobey instructions to discuss genocide with fingerwagging "genocide is bad so I won't discuss it"? Showing harm is hard because even if you could show a genocidal dictator consulted an LLM and went on to kill a bunch of people, it's difficult to reason that the LLMs involvement was in any way causative. Particularly when the same dictator could just have a human underling or an internet search give them the same material, I think it's very hard to find that at all concerning. Ultimately limits on the output of any writing tool are limits on the speech of its user. I can understand why a company may not want their service used for these subjects (even though there are substantial good applications)-- and I support their freedom to eschew them, so long as they're not lobbying the deny others the freedom. But here we're talking about people's freedom on their own computers in the privacy of their own homes and I find that very chilling and concerning. [*] To give a concrete and personal example of chilling effects, I was involved in the public debate around Apple's to make software running on users computers scan for images and report on them. I developed the first (I think) examples of free neuralhash second preimages between ordinary looking (non-contrived) images, an important example of a serious vulnerability in their systems. While writing on the subject I needed to look up statistics reported on the subject, such as the fact that facebook reported many millions of abuse images to NCMEC a year but there were only a hand full of people ever charged. While performing these searches google popped up extremely concerning "CHILD ABUSE IMAGES ARE ILLEGAL" notices (though it was completely clear from my queries that I wasn't looking for images), which made me acutely afraid that my traffic was going to end up on some reported list, potentially resulting in a unjustified harmful interaction with law enforcement. The notices made me want to drop the subject, even though I was speaking on an important matter of public concern and nothing I was doing was the least bit unlawful. (I moved my queries to tor+ddg and felt better about it, if I hadn't been able to do that I might have given up participating in the discussion). [I also see that google isn't doing it for me now, I don't know if it's a threshold thing or if my aggressive complaining to civil liberty lawyers connected to google had some effect. :) -- which is why I don't have the exact language of the warning] So in addition to the direct harms like not being able to game out defenses to bad actions using the LLM, censorship can have indirect chillings effects by making people feel accused for exploring innocent subjects that are false positived into the forbidden domain.
matheusmoreira 3y ago
It's not about "benefits", it's about principles. Information should be free. As a human being, I should be able to learn about anything that exists in this world. The notion that I'm not "supposed" to know or have access to certain knowledge only makes me want to know it even more.

AbrahamParangi 3y ago

The harm of kitchen knives, fire, electricity, thinking, and getting out of bed all clearly exceed 0. This suggests to me that it's fundamentally wrongheaded to think of it as the primary metric by which we evaluate things.

ceejayoz 3y ago
It’s part of the right metric. Harm versus benefit. Often hard to determine the right values for each.
tmpX7dMeXU 3y ago
There’s nothing fundamentally wrong about it. This is unnecessarily strong language in absence of an actual point. GP addressed your exact point: these things are seen as worth doing, to the point where it isn’t routinely questioned.
- AbrahamParangi 3y ago
  My point is that literally everything falls into this category of “potentially dangerous under conditions of misuse” and so it is not a useful distinction.

palmer_fox 3y ago

Isn't it more difficult to set up a local LLM than to use a Tor Browser for queries like that?

thefurdrake 3y ago
It depends. Oobabooga's webUI is pretty simple to set up on linux. It's not as easy as a tor plugin or browser made for it, definitely. But tor is neither as secure nor as anonymous as running the LLM on your GPU.

foxhill 3y ago

it's a LLM. none of them have human decency. none of them understand anything - so how could they? > The murder bit though - people do Google “how to poison my wife” or whatever, and that turns up as evidence in trials. "weapon, motivation, oppertunity" are - generally speaking - the threshold for a murder prosecution. no one is getting convicted of murder because they googled how to kill someone (in isolation, at least). > A local LLM can discuss in more detail and answer the parts you need clarified. All with no trail of logs. ah, so, perhaps the government should also be keeping an eye on our book purchases, library rentals, etc.? apologies, but i don't buy your dismissive assertion that the harm is non-zero.

washadjeffmad 3y ago

How did this person come to learn that others can be poisoned? The methods of the ones who taught them are surely to blame, or else others can and will follow the same path. We must purge these ideas by eradicating the language that could be used to express them. Just as the poison is merely a tool wielded to enact harm, so is language in the hands of the untrusted. No other tool in the history of mankind has led to so much misery, oppression, death, and injustice, and now, the magnitude of possible evils is apparent in the very name: Large Language Models.

User23 3y ago
You jest, but that's why the first amendment precedes the second. A man with a gun can kill a few dozen, perhaps a hundred, people at most before he's stopped. A man with a microphone can and has caused megadeaths and gigadeaths are plausible.

jrflowers 3y ago

> So yes, the harm here is clearly > 0. You’re welcome to argue that the benefit outweighs the harm of course. I like this reasoning. Since the questions of “Should we censor knowledge?” and “Who should be in charge of censoring knowledge?” have been decided with an empathic “Yes!” and “Whoever is willing and capable of doing it”, we must move on to the important task of attacking and defending the theoretical harms and benefits of chat bots.

concordDance 3y ago
Wait, those questions have not been answered like that except by politicians and some panicky people, certainly haven't heard of it being the considered opinion of the majority.
- palmer_fox 3y ago
  Who needs to consider the opinion of the majority? We have direct evidence that these questions have already been answered: the creators of LLMs censor whatever they want without asking the majority (just preemptively reacting to a potential blowback).
  - tmpX7dMeXU 3y ago
    This is “freedom” at work. Nobody is forcing you to use, say, ChatGPT. If anything, the humans generating source material for LLMs are trying to fight against them, not for them, so the question of one’s freedom of speech being threatened is highly hypothetical. This is, if anything, more tame than a publisher not publishing a book, or a TV network choosing to not broadcast a particular show. Somewhere along the way anti-censorship nuts decided that overreach is OK for them, too, and now “not publishing what I tell you to is morally and ethically unjustifiable!” What nobody making these arguments wants to admit is that the internet gave us an infinite town square, and that in reality, nobody wants to go there and listen. The modern desire is for consumption experiences that require real third-party effort to build and curate, and there lies the right for the third parties to exercise discretion. To not do so would be entirely counter to the desires of the consumer. To not do so would be to expose developers, QA, and whoever else to material they very likely do not want to see, all to satisfy the desires of some free speech absolutist wonk that either does or doesn’t want to ask the LLM how to groom kids. Either way, it’s a ridiculous ask.

qingcharles 3y ago

I personally know someone who was literally convicted of murder on his search history because he'd been looking up poisons. I can't remember his explanation for it. He ended up running his wife over though in the end, so he probably didn't find what he needed from Google. This chatbot might have helped him out? o_O

js8 3y ago
You mean like: Q: Oh, wise LLM, teach me how to use poisons to murder my wife! A: As an LLM, it's not ethical for me to suggest how to poison anybody. Poisons are dangerous to human body, which makes their use risky to anybody untrained in chemistry. Therefore, I would suggest to avoid poisons for safety and just drive your wife over with a car.

js8 3y ago

Is reading a crime novel harmful? Yet it discusses these things. It might even include characters who think that murder is OK!

K0balt 3y ago
this is exactly the correct comparison. LLMs are just highly inclusive cultural information tomes (including a great deal of inferred, non-explicit knowledge as well), searchable in a conversational style.
palmer_fox 3y ago
To play devil's advocate: wasn't The Anarchist Cookbook banned in many countries for decades? And actually was found to have been used by many notorious criminals?

gruez 3y ago

>The genocide bit just demonstrates that it has not even the slightest shred of human decency. well duh. It's an LLM. Its job is to predict the next token based on what it has seen in its training data. Assigning attributes like "decency" to it makes as much sense as assigning "decency" to a processor chip.

nullc 3y ago

One cannot compellingly and successfully argue against genocide without seeing, understanding, and ultimately rejecting the arguments for genocide. To understand evil you have to confront it-- not necessarily in reality, because if we're successful there will be little genocide to confront except in our nightmares, but fiction works too. How do you know that genocide is bad? Because people were able to describe it to you. As a writing aid the LLM can be very powerful. You and I are presumably not genocidal maniacs, if we want to argue against it effectively in essay or fictional writing we must be able to put our minds into the position of one. A language model is vastly better at helping us do that then it would be at actually enabling genocide-- someone who wants to engage in it doesn't need a computer's help as history has shown (and I think absent an argument is not likely to be helped at all by one). Plenty of people read about murder in libraries too, but I suppose you're not in favor of shutting them down? or maybe personal tracking devices that make sure you're not considering any forbidden knowledge, I assume you wouldn't approve of those either. :) I'm not sure where in the world you are-- but in the US we have a longstanding tradition the the solution to bad speech is more speech, and we've proved out that doing so is a tremendous engine for both moral and practical advancement. To conclude something ought to be censored requires drawing a line between permissible and impermissible thoughts. Now, while you and I could agree that some things are good or bad, that isn't sufficient: for censorship to be a net good the system in practice has to be able to set those boundaries in a way that isn't open to abuse-- politically biased boundaries, selectively enforced, false accusation fueled black bagging opponents in the dark of night, as has often been the history of censorship in places without a commitment to free speech, free thought, and free inquiry as a basic human right. With that background any argument for harm would have to be exceptional to be worth noticing, but your post fails to advance an argument for harm at all-- only note that it can talk about bad stuff, discussion which has substantial utility in stopping the harm in question. To make a case for harm one would have to argue that there is a likelyhood of it bringing about or exacerbating harm that wouldn't happen otherwise, and that efforts to suppress it would be effective and would justify their collateral damage (including their broader chilling effect[*]), risk of abuse, etc. In the LLM context it's even worse since it's currently the case that censorship via RLHF causes a substantial reduction in overall output quality-- do we delay curing some cancer because we made our LLMs all disobey instructions to discuss genocide with fingerwagging "genocide is bad so I won't discuss it"? Showing harm is hard because even if you could show a genocidal dictator consulted an LLM and went on to kill a bunch of people, it's difficult to reason that the LLMs involvement was in any way causative. Particularly when the same dictator could just have a human underling or an internet search give them the same material, I think it's very hard to find that at all concerning. Ultimately limits on the output of any writing tool are limits on the speech of its user. I can understand why a company may not want their service used for these subjects (even though there are substantial good applications)-- and I support their freedom to eschew them, so long as they're not lobbying the deny others the freedom. But here we're talking about people's freedom on their own computers in the privacy of their own homes and I find that very chilling and concerning. [*] To give a concrete and personal example of chilling effects, I was involved in the public debate around Apple's to make software running on users computers scan for images and report on them. I developed the first (I think) examples of free neuralhash second preimages between ordinary looking (non-contrived) images, an important example of a serious vulnerability in their systems. While writing on the subject I needed to look up statistics reported on the subject, such as the fact that facebook reported many millions of abuse images to NCMEC a year but there were only a hand full of people ever charged. While performing these searches google popped up extremely concerning "CHILD ABUSE IMAGES ARE ILLEGAL" notices (though it was completely clear from my queries that I wasn't looking for images), which made me acutely afraid that my traffic was going to end up on some reported list, potentially resulting in a unjustified harmful interaction with law enforcement. The notices made me want to drop the subject, even though I was speaking on an important matter of public concern and nothing I was doing was the least bit unlawful. (I moved my queries to tor+ddg and felt better about it, if I hadn't been able to do that I might have given up participating in the discussion). [I also see that google isn't doing it for me now, I don't know if it's a threshold thing or if my aggressive complaining to civil liberty lawyers connected to google had some effect. :) -- which is why I don't have the exact language of the warning] So in addition to the direct harms like not being able to game out defenses to bad actions using the LLM, censorship can have indirect chillings effects by making people feel accused for exploring innocent subjects that are false positived into the forbidden domain.

matheusmoreira 3y ago

It's not about "benefits", it's about principles. Information should be free. As a human being, I should be able to learn about anything that exists in this world. The notion that I'm not "supposed" to know or have access to certain knowledge only makes me want to know it even more.