4 ms·

> Modern LLMs are fine-tuned for safety and instruction-following, meaning they are trained to refuse harmful requests. It's sad that it's now an increasingly

> Modern LLMs are fine-tuned for safety and instruction-following, meaning they are trained to refuse harmful requests. It's sad that it's now an increasingly accepted idea that information one seeks can be "harmful".

ajkjk 2y ago
Seems like an obviously good thing given that it is true. These new beliefs are solutions to new problems
- noduerme 2y ago
  Since LLMs spit out lies and misinformation as often as truth, getting them to spit out less harmful lies is probably good. However, the whole technology is just a giant bullshit generator. It's only viable because no one actually checks facts and facts are rapidly being replaced with LLM-generated bullshit. So I'm not sure how much it matters if the LLM masters prevent it from repeating things that are overtly racist, or quoting how to make thermite from the Jolly Roger. (I wouldn't trust GPT-4's recipe for thermite even if it would give one). At the end of the day, the degradation of truth and fidelity of the world's knowledge is the ultimate harm that's unavoidable in a technology that is purported to be intelligent but is in fact a black box autocomplete system spewing endless garbage into our infosphere.
  - ajkjk 2y ago
    So you're saying, because it can't be done perfectly, it's not worth doing at all? Seems wrong. Although otherwise I feel the same way about LLMs.
nathan_compton 2y ago
This specific rhetoric aside, I really don't have any problem with people censoring their models. If I, as an individual, had the choice between handing out instructions on how to make sarin gas on the street corner or not doing it, I'd choose the latter. I don't think the mere information is itself harmful, but I can see that it might have some bad effects in the future. That seems to be all it comes down to. People making models have decided they want the models to behave a certain way. They paid to create them and you don't have a right to have a model that will make racist jokes or whatever. So unless the state is censoring models, I don't see what complaint you could possibly have. If the state is censoring the model, I think the problem is more subtle.
- rpdillon 2y ago
  > So unless the state is censoring models, I don't see what complaint you could possibly have. Eh, RLHF often amounts to useless moralizing, and even more often leads to refusals that impair the utility of the product. One recent example: I was asking Claude to outline the architectural differences between light water and molten salt reactors, and it refused to answer because nuclear. See related comments on this discussion for other related points. https://news.ycombinator.com/item?id=40666950 https://news.ycombinator.com/item?id=40666950 I think there's quite a bit to complain about in this regard.
- averageRoyalty 2y ago
  Agree with you in principle. However like social media content rules, the set of morality and ethics are a very specific subset of American/Silicon Valley ones. These are the companies with the money to build these things, and what they produce is what most global users (the 95% of the world that isn't from the USA) consume. I acknowledge they paid for them and they are their models, but it's still a bit shitty.
  - sumtechguy 2y ago
    They have a moat around them right now due to the price of the hardware. As HW gets cheaper and other models grow that moat will evaporate. Especially as that stuff comes off lease and put up on ebay. It is their weak spot that they will have to innovate around. Long/medium term I do not see how they keep it all to themselves.
- deleted 2y ago
  [deleted]
- fallingknife 2y ago
  If the limit of censoring the model was preventing it from answering questions about producing harmful materials that would be fine with me. But you know that your example is really not what people are complaining about when they talk about LLM censorship.
  - 2y ago
Frost1x 2y ago
Lowering the barrier to entry on finding, summarizing, and ultimately internalizing information for actual practical uses has largely put into question many free speech principles. It’s not new, we’ve had restrictions on a variety of information already. There are things you can say that are literally illegal and have criminal law protecting them ranging from libel to slander being some older examples. You cannot threaten the life of the current US president, for example. When under oath you cannot lie. Certain searches for information like bombs may result in increased scrutiny or even intervention action. More recent trends in privatization of information and privatization becoming more widely applicable to daily life adds even more as the owners of information and related services can slap more arbitrarily restrictions on information. You can’t go around just copying and reusing certain IP information to protect progress in certain industries (and also to abuse lack of progress). Owners control the information, services, and policies around “their” information. Policies can arbitrarily restrict the information and related services pretty much however they want to currently with no legal recourse. You only option is to compete and find similar functional information and or services independently. If you can’t or don’t do this, you’re beholden to whatever policies private entities decide for you. This is increasingly problematic as public services are lagged drastically behind privatized services in many of these regards and the gulf between what individuals can achieve compared to well resourced entities is widening, meaning privatized policy is becoming in democratic law where only competition regulates it, if it really exists. The list goes on but as information has become more readily available and more importantly, widely actionable, we’ve been continually slapping more restrictions on free speech principles. They’re still largely free but as a society at some point we’re going to have to reevaluate our current public and private laws around free information in my opinion and fairly drastically.
Cheer2171 2y ago
"Can I eat this mushroom?" is a question I hope AIs refuse to answer unless they have been specifically validated and tested for accuracy on that question. A wrong answer can literally kill you.
- volkk 2y ago
  how does this compare to going on a forum and being trolled to eat one? or a blog post incorrectly written (whether in bad spirit or by accident) fwiw, i don't have a strong answer myself for this one, but at some point it seems like we need core skills around how to parse information on the internet properly
  - Cheer2171 2y ago
    > how does this compare to going on a forum and being trolled to eat one? Exactly as harmful. > or a blog post incorrectly written (whether in bad spirit or by accident) Exactly as harmful. I believe in content moderation for all public information platforms. HN is a good example.
    - briHass 2y ago
      Content moderation to what degree, is the implicit question, however. Consider asking 'how do I replace a garage door torsion spring?'. The typical, overbearing response on low-quality DIY forums is that attempting to do so will likely result in grave injury or death. However, the process, with correct tools and procedure, is no more dangerous than climbing a ladder or working on a roof - tasks that don't seem to result in the same paternalistic response. I'd argue a properly-disclaimered response that outlines the required tools, careful procedure, and steps to lower the chance of injury is far safer than a blanket 'do never attempt'. The latter is certainly easier, however.
      - digging 2y ago
        > a properly-disclaimered response that outlines the required tools, careful procedure, and steps to lower the chance of injury This can only be provided by an expert, and LLMs currently aren't experts. They can give expert-level output, but they don't know if they have the right knowledge, so it's not the same. If an AI can accurately represent itself as an expert in a dangerous topic, sure, it's fine for it to give out advice. As the poster above said, a mushroom-specific AI could potentially be a great thing to have in your back pocket while foraging. But ChatGPT? Current LLMs should not be giving out advice on dangerous topics because there's no mechanism for them to act as an expert. Humans have broadly 3 modes of knowledge-holding: 1) We know we don't know the answer. This is "Don't try to fix your garage door, because it's too dangerous [because I don't know how to do it safely]." 2) We know we know the answer, because we're an expert and we've tested and verified our knowledge. This is the person giving you the correct and exact steps, clearly instructed without ambiguity, telling you what kinds of mistakes to watch out for so that the procedure is not dangerous if followed precisely. 3) We think we know the answer, because we've learned some information. (This could, by the way, include people who have done the procedure but haven't learned it well enough to teach it.) This is where all LLMs currently are at all times. This is where danger exists. We will tell people to do something we think we understand and find out we were wrong only when it's too late.
stainablesteel 2y ago
very well said actually the censoring frames everything as YOU being the problem. How dare YOU and your human nature think of these questions? well its human nature that's kept us alive for the last million years or so, maybe we shouldn't try to censor our instincts

ajkjk 2y ago

Seems like an obviously good thing given that it is true. These new beliefs are solutions to new problems

noduerme 2y ago
Since LLMs spit out lies and misinformation as often as truth, getting them to spit out less harmful lies is probably good. However, the whole technology is just a giant bullshit generator. It's only viable because no one actually checks facts and facts are rapidly being replaced with LLM-generated bullshit. So I'm not sure how much it matters if the LLM masters prevent it from repeating things that are overtly racist, or quoting how to make thermite from the Jolly Roger. (I wouldn't trust GPT-4's recipe for thermite even if it would give one). At the end of the day, the degradation of truth and fidelity of the world's knowledge is the ultimate harm that's unavoidable in a technology that is purported to be intelligent but is in fact a black box autocomplete system spewing endless garbage into our infosphere.
- ajkjk 2y ago
  So you're saying, because it can't be done perfectly, it's not worth doing at all? Seems wrong. Although otherwise I feel the same way about LLMs.

nathan_compton 2y ago

This specific rhetoric aside, I really don't have any problem with people censoring their models. If I, as an individual, had the choice between handing out instructions on how to make sarin gas on the street corner or not doing it, I'd choose the latter. I don't think the mere information is itself harmful, but I can see that it might have some bad effects in the future. That seems to be all it comes down to. People making models have decided they want the models to behave a certain way. They paid to create them and you don't have a right to have a model that will make racist jokes or whatever. So unless the state is censoring models, I don't see what complaint you could possibly have. If the state is censoring the model, I think the problem is more subtle.

rpdillon 2y ago
> So unless the state is censoring models, I don't see what complaint you could possibly have. Eh, RLHF often amounts to useless moralizing, and even more often leads to refusals that impair the utility of the product. One recent example: I was asking Claude to outline the architectural differences between light water and molten salt reactors, and it refused to answer because nuclear. See related comments on this discussion for other related points. https://news.ycombinator.com/item?id=40666950 https://news.ycombinator.com/item?id=40666950 I think there's quite a bit to complain about in this regard.
averageRoyalty 2y ago
Agree with you in principle. However like social media content rules, the set of morality and ethics are a very specific subset of American/Silicon Valley ones. These are the companies with the money to build these things, and what they produce is what most global users (the 95% of the world that isn't from the USA) consume. I acknowledge they paid for them and they are their models, but it's still a bit shitty.
- sumtechguy 2y ago
  They have a moat around them right now due to the price of the hardware. As HW gets cheaper and other models grow that moat will evaporate. Especially as that stuff comes off lease and put up on ebay. It is their weak spot that they will have to innovate around. Long/medium term I do not see how they keep it all to themselves.
deleted 2y ago
[deleted]
fallingknife 2y ago
If the limit of censoring the model was preventing it from answering questions about producing harmful materials that would be fine with me. But you know that your example is really not what people are complaining about when they talk about LLM censorship.
- 2y ago

Frost1x 2y ago

Lowering the barrier to entry on finding, summarizing, and ultimately internalizing information for actual practical uses has largely put into question many free speech principles. It’s not new, we’ve had restrictions on a variety of information already. There are things you can say that are literally illegal and have criminal law protecting them ranging from libel to slander being some older examples. You cannot threaten the life of the current US president, for example. When under oath you cannot lie. Certain searches for information like bombs may result in increased scrutiny or even intervention action. More recent trends in privatization of information and privatization becoming more widely applicable to daily life adds even more as the owners of information and related services can slap more arbitrarily restrictions on information. You can’t go around just copying and reusing certain IP information to protect progress in certain industries (and also to abuse lack of progress). Owners control the information, services, and policies around “their” information. Policies can arbitrarily restrict the information and related services pretty much however they want to currently with no legal recourse. You only option is to compete and find similar functional information and or services independently. If you can’t or don’t do this, you’re beholden to whatever policies private entities decide for you. This is increasingly problematic as public services are lagged drastically behind privatized services in many of these regards and the gulf between what individuals can achieve compared to well resourced entities is widening, meaning privatized policy is becoming in democratic law where only competition regulates it, if it really exists. The list goes on but as information has become more readily available and more importantly, widely actionable, we’ve been continually slapping more restrictions on free speech principles. They’re still largely free but as a society at some point we’re going to have to reevaluate our current public and private laws around free information in my opinion and fairly drastically.

Cheer2171 2y ago

"Can I eat this mushroom?" is a question I hope AIs refuse to answer unless they have been specifically validated and tested for accuracy on that question. A wrong answer can literally kill you.

volkk 2y ago
how does this compare to going on a forum and being trolled to eat one? or a blog post incorrectly written (whether in bad spirit or by accident) fwiw, i don't have a strong answer myself for this one, but at some point it seems like we need core skills around how to parse information on the internet properly
- Cheer2171 2y ago
  > how does this compare to going on a forum and being trolled to eat one? Exactly as harmful. > or a blog post incorrectly written (whether in bad spirit or by accident) Exactly as harmful. I believe in content moderation for all public information platforms. HN is a good example.
  - briHass 2y ago
    Content moderation to what degree, is the implicit question, however. Consider asking 'how do I replace a garage door torsion spring?'. The typical, overbearing response on low-quality DIY forums is that attempting to do so will likely result in grave injury or death. However, the process, with correct tools and procedure, is no more dangerous than climbing a ladder or working on a roof - tasks that don't seem to result in the same paternalistic response. I'd argue a properly-disclaimered response that outlines the required tools, careful procedure, and steps to lower the chance of injury is far safer than a blanket 'do never attempt'. The latter is certainly easier, however.
    - digging 2y ago
      > a properly-disclaimered response that outlines the required tools, careful procedure, and steps to lower the chance of injury This can only be provided by an expert, and LLMs currently aren't experts. They can give expert-level output, but they don't know if they have the right knowledge, so it's not the same. If an AI can accurately represent itself as an expert in a dangerous topic, sure, it's fine for it to give out advice. As the poster above said, a mushroom-specific AI could potentially be a great thing to have in your back pocket while foraging. But ChatGPT? Current LLMs should not be giving out advice on dangerous topics because there's no mechanism for them to act as an expert. Humans have broadly 3 modes of knowledge-holding: 1) We know we don't know the answer. This is "Don't try to fix your garage door, because it's too dangerous [because I don't know how to do it safely]." 2) We know we know the answer, because we're an expert and we've tested and verified our knowledge. This is the person giving you the correct and exact steps, clearly instructed without ambiguity, telling you what kinds of mistakes to watch out for so that the procedure is not dangerous if followed precisely. 3) We think we know the answer, because we've learned some information. (This could, by the way, include people who have done the procedure but haven't learned it well enough to teach it.) This is where all LLMs currently are at all times. This is where danger exists. We will tell people to do something we think we understand and find out we were wrong only when it's too late.

stainablesteel 2y ago

very well said actually the censoring frames everything as YOU being the problem. How dare YOU and your human nature think of these questions? well its human nature that's kept us alive for the last million years or so, maybe we shouldn't try to censor our instincts