5 ms·

It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.

by ozr 3y ago

goatlover 3y ago
Indeed. Why do they get to decide for humanity?
- maxbond 3y ago
  Because they're private corporations and these models are their private property, indeed they are a type of capital like a bottling machine or a loom. This isn't so much about moralizing as much as it is businesses deciding what to do to make the most money. That doesn't mean you can't disagree with it, far from it. But I think the framing of, "these companies are imposing their morality on me" is a misdiagnosis. I don't think it's really a moral position for them, it's a product engineering position. I would describe the situation as, "more and more of the world is controlled by large corporations, and I'm increasingly subject to their arbitrary and unaccountable decisions. Many of which make no sense from my vantage point."
- yeck 3y ago
  They are proving that the present techniques being used to control their models can be reliably bypassed. Regardless of what you want your model to be able to do, there may be things that you don't want it to do. For instance, if you are making a product for children you'd probably want it to avoid certain content. If you are training you own model, it would be nice to know what, if any, techniques you could employ to balance the effectiveness of it with generality.
thisisthenewme 3y ago
Is the issue you have with the group of people doing the moderation, or with the idea of the moderation in the first place? Are you certain that its the 'SV engineers' that are doing the current moderation? If you think the problem is with the current group of moderators, who do you think should be moderating and what should be the criteria of their moderation? If you think we don't need any moderation, do you believe that people should have a fairly easy access to * Learn how to make bombs (as mentioned in the article) * Get away with committing crimes? Are moderating these topics related to morality?
- int_19h 3y ago
  When the "crimes" in question are e.g. drug use or abortion, yes, moderating these topics is very much related to morality.
- amluto 3y ago
  It’s not exactly difficult to find the resources from which the LLMs probably learned these answers in the first place. I can think of many things an LLM could do that would be far more harmful than any of this.
- casefields 3y ago
  The Progressive laid out how to build a hydrogen bomb: https://progressive.org/magazine/november-1979-issue/ https://progressive.org/magazine/november-1979-issue/ The US government said that info was born secret and sued: https://en.wikipedia.org/wiki/United_States_v._Progressive,_Inc https://en.wikipedia.org/wiki/United_States_v._Progressive,_.... I won’t spoil who won the argument.
- JoeyBananas 3y ago
  In the United States, I can write a pamphlet about getting away with crimes and making bombs and hand it out on the street. There is nothing inherently illegal about those topics.
  - fbdab103 3y ago
    Just don't talk about jury nullification[0] [0]https://www.usnews.com/news/articles/2017-06-02/jury-convicts-ex-pastor-who-shared-jury-nullification-fliers https://www.usnews.com/news/articles/2017-06-02/jury-convict...
    - nradov 3y ago
      That conviction was overturned. https://fija.org/news-events/2020/july/keith-wood-conviction-overturned-by-michigan-supreme-court.html https://fija.org/news-events/2020/july/keith-wood-conviction...
yeck 3y ago
Sure, these particular examples are concerned with morality, but the problem is more general and limits the value of language models because they can be hacked for other purposes. A good example that's been going around is having an agential model that manages you emails. Someone sends you an email using prompt injection to compel the agent to delete all your emails. Or forward all you emails to another address. If there isn't a way to secure the behaviour of AI models against reliable exploits then the utility of the models is dramatically limited.
- TheAceOfHearts 3y ago
  Use multiple models or gap their capabilities.
- bick_nyers 3y ago
  Don't allow it to delete emails (maybe allow mark for delete in 30 days?) and have a whitelist of acceptable forwarding addresses or push a confirm/deny notification to a manual reviewer. It's like AI diagnosis, we aren't going to run it full stop automated without safeguards on top or manual review for a long time.

It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.

by ozr 3y ago

It's not a 'vulnerability'. It's allowing people to use the models without the morals of a small number of SV engineers being impressed on you.

goatlover 3y ago

Indeed. Why do they get to decide for humanity?

maxbond 3y ago
Because they're private corporations and these models are their private property, indeed they are a type of capital like a bottling machine or a loom. This isn't so much about moralizing as much as it is businesses deciding what to do to make the most money. That doesn't mean you can't disagree with it, far from it. But I think the framing of, "these companies are imposing their morality on me" is a misdiagnosis. I don't think it's really a moral position for them, it's a product engineering position. I would describe the situation as, "more and more of the world is controlled by large corporations, and I'm increasingly subject to their arbitrary and unaccountable decisions. Many of which make no sense from my vantage point."
yeck 3y ago
They are proving that the present techniques being used to control their models can be reliably bypassed. Regardless of what you want your model to be able to do, there may be things that you don't want it to do. For instance, if you are making a product for children you'd probably want it to avoid certain content. If you are training you own model, it would be nice to know what, if any, techniques you could employ to balance the effectiveness of it with generality.

thisisthenewme 3y ago

Is the issue you have with the group of people doing the moderation, or with the idea of the moderation in the first place? Are you certain that its the 'SV engineers' that are doing the current moderation? If you think the problem is with the current group of moderators, who do you think should be moderating and what should be the criteria of their moderation? If you think we don't need any moderation, do you believe that people should have a fairly easy access to * Learn how to make bombs (as mentioned in the article) * Get away with committing crimes? Are moderating these topics related to morality?

int_19h 3y ago
When the "crimes" in question are e.g. drug use or abortion, yes, moderating these topics is very much related to morality.
amluto 3y ago
It’s not exactly difficult to find the resources from which the LLMs probably learned these answers in the first place. I can think of many things an LLM could do that would be far more harmful than any of this.
casefields 3y ago
The Progressive laid out how to build a hydrogen bomb: https://progressive.org/magazine/november-1979-issue/ https://progressive.org/magazine/november-1979-issue/ The US government said that info was born secret and sued: https://en.wikipedia.org/wiki/United_States_v._Progressive,_Inc https://en.wikipedia.org/wiki/United_States_v._Progressive,_.... I won’t spoil who won the argument.
JoeyBananas 3y ago
In the United States, I can write a pamphlet about getting away with crimes and making bombs and hand it out on the street. There is nothing inherently illegal about those topics.
- fbdab103 3y ago
  Just don't talk about jury nullification[0] [0]https://www.usnews.com/news/articles/2017-06-02/jury-convicts-ex-pastor-who-shared-jury-nullification-fliers https://www.usnews.com/news/articles/2017-06-02/jury-convict...
  - nradov 3y ago
    That conviction was overturned. https://fija.org/news-events/2020/july/keith-wood-conviction-overturned-by-michigan-supreme-court.html https://fija.org/news-events/2020/july/keith-wood-conviction...

yeck 3y ago

Sure, these particular examples are concerned with morality, but the problem is more general and limits the value of language models because they can be hacked for other purposes. A good example that's been going around is having an agential model that manages you emails. Someone sends you an email using prompt injection to compel the agent to delete all your emails. Or forward all you emails to another address. If there isn't a way to secure the behaviour of AI models against reliable exploits then the utility of the models is dramatically limited.

TheAceOfHearts 3y ago
Use multiple models or gap their capabilities.
bick_nyers 3y ago
Don't allow it to delete emails (maybe allow mark for delete in 30 days?) and have a whitelist of acceptable forwarding addresses or push a confirm/deny notification to a manual reviewer. It's like AI diagnosis, we aren't going to run it full stop automated without safeguards on top or manual review for a long time.