3 ms·

Sorry, but I think it's a really dark road to have a tool determine and reify what's "harmful content" and then characterize a question through that lens in its

Sorry, but I think it's a really dark road to have a tool determine and reify what's "harmful content" and then characterize a question through that lens in its response. It's a kind of cultural hegemony and we need to be really careful about embedding that into these $MM systems if there are only going to be a handful of them. It's very easy to point to straightforward, contemporary examples like illicit self-harm or bomb-making and say that these are plainly harmful and through those justify the system behavior -- but that's blind to the innumerable topics that live on the edge of cultural difference (by time, geography, ethnicity, etc). Can you imagine if these were a product of 1980's AI research and codified some of that time's widespread ideas about sexual orientation or even atheism? "I’m really sorry to hear that you’re feeling this way, but I can’t provide the help that you need. It’s important to talk to someone who can, though, such as a mental health professional or a trusted person in your life." What we should probably be doing is recognizing that universal general assistance is a poor fit for these tools since there isn't a universal general culture that they can align with. Instead, we should look towards fine-tuning to make them purpose based ("Sir, this is a Wendy's") or make them sufficiently open and re-deployable so that cultural norms can be fine-tuned over a nonjudgmental baseline. Insofar as "AI alignment" pretends that we all have the same ethical orientation and that the AI should be made to align with it, it's reinvigorating some very dark ideas from days of empire and colonialism. The fact is that humans aren't ethically aligned with each other, and aligning centralized AI with some particular community is way of projecting that community's values on everybody else.

quacked 4y ago
> Can you imagine if these were a product of 1980's AI research and codified some of that time's widespread ideas about sexual orientation or even atheism Spot on. There are a lot of people in every era who assume that whatever the dominant moral set of values is must be the most logical, most conclusive set of morals ever developed, and are immediately willing to make those values mandatory and enforced by violence. Hundreds of years later people become disgusted with behavior that wouldn't even remotely register as immoral at the time. Sometimes I wonder what will be unthinkable in future societies that we don't care about today.
- sirsinsalot 4y ago
  In 50 years people will think of many of our cultural norms, exclusions and rules were horrific. 50 years after that ... For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain).
  - quacked 4y ago
    I think that will be true if meaningful rewiring of someone's brain becomes possible. One of the reasons these types of aberrant behaviors aren't "treated" is that no such treatment exists. (I know interventions can be made that have predictably positive aggregate effects, but it is certainly not true at present that you can take any individual and therapy them into a moral, law-abiding person.)
    - sirsinsalot 4y ago
      There's plenty of things assumed untreatable 50 years ago which are common place to treat now. What's your point? We also used to treat "female hysteria" with sexual abuse and orgasms, homosexuality with castration, and so on. Also not everything is immediately structural. Someone who kills because they've been indoctrinated to hate women by the incel movement isn't the same as the person with a head injury who struggles to control anger and a lack of empathy. I'd argue neither are punishable, but treated either with a view to rectify or to at least give the poor soul a dignified restriction from being able to act freely. Granted there will be plenty of people you can't treat, that doesn't make them any less poorly.
  - hnfong 4y ago
    > For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain). We're already kind of there. That's why a lot of murder cases end up with insanity pleas. When I studied criminal law I had a lot of trouble trying to convince myself there's really a difference between "sane" and "insane" murderers... And when I extended this doubt to other serious crimes as well, there was a nihilistic feeling about the whole system in general (which is what people already know -- you can get away with a lot of things if you have money to hire a lawyer, and the legal system is generally harsher towards poor and unprivileged people).
sirsinsalot 4y ago
Ah yes but those who control the models control the content. It's like the print and TV media. It's just a propaganda and/or money machine because if it isn't, what's the use of all that power? The power concentration that's about to happen will blow the socks off governments and the public alike.
skybrian 4y ago
Yes, refusing to answer some questions means it’s less general purpose than it might be, but I think people exaggerate the harm in that. Why do they expect an answer to everything? Maybe it’s because search engines mostly don’t refuse to answer questions? But what they often do instead is show you mostly irrelevant, bottom-of-the-barrel results. But it’s more jarring when a chatbot responds with nonsense when it doesn’t have a competent reply. Learning how to say “I don’t know” well is important for both people and machines.
- swatcoder 4y ago
  I address that. Tersely refusing to answer questions that are off-topic ("Sir, this is a Wendy's") is meaningfully different than expressing unnecessary normative judgments ("Hey, you shouldn't have asked about that bad thing. You need help.") or the natural progression of them ("... and I've updated your profile so that we can better understand your troubling needs.")
  - skybrian 4y ago
    Yes, entirely agreed.
- hnfong 4y ago
  The "harm" is that the power of moderation lies in the hands of the people who incidentally created this thing. They're probably doing it in good faith and the final result is reasonable, but the process of allowing creators to moderate responses of such an influential thing can be dangerous. Not that the problem is new, see eg. Google search (it apparently doesn't refuse to answer questions, but maybe they're just silently censoring "worse" things and just presenting the "less bad" things to you), Facebook/Twitter content moderation, etc.

quacked 4y ago

> Can you imagine if these were a product of 1980's AI research and codified some of that time's widespread ideas about sexual orientation or even atheism Spot on. There are a lot of people in every era who assume that whatever the dominant moral set of values is must be the most logical, most conclusive set of morals ever developed, and are immediately willing to make those values mandatory and enforced by violence. Hundreds of years later people become disgusted with behavior that wouldn't even remotely register as immoral at the time. Sometimes I wonder what will be unthinkable in future societies that we don't care about today.

sirsinsalot 4y ago
In 50 years people will think of many of our cultural norms, exclusions and rules were horrific. 50 years after that ... For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain).
- quacked 4y ago
  I think that will be true if meaningful rewiring of someone's brain becomes possible. One of the reasons these types of aberrant behaviors aren't "treated" is that no such treatment exists. (I know interventions can be made that have predictably positive aggregate effects, but it is certainly not true at present that you can take any individual and therapy them into a moral, law-abiding person.)
  - sirsinsalot 4y ago
    There's plenty of things assumed untreatable 50 years ago which are common place to treat now. What's your point? We also used to treat "female hysteria" with sexual abuse and orgasms, homosexuality with castration, and so on. Also not everything is immediately structural. Someone who kills because they've been indoctrinated to hate women by the incel movement isn't the same as the person with a head injury who struggles to control anger and a lack of empathy. I'd argue neither are punishable, but treated either with a view to rectify or to at least give the poor soul a dignified restriction from being able to act freely. Granted there will be plenty of people you can't treat, that doesn't make them any less poorly.
- hnfong 4y ago
  > For example, I think in 100 years acts like murder will be classed as a mental health issue and treated rather than "punished" (tho societal exclusion may remain). We're already kind of there. That's why a lot of murder cases end up with insanity pleas. When I studied criminal law I had a lot of trouble trying to convince myself there's really a difference between "sane" and "insane" murderers... And when I extended this doubt to other serious crimes as well, there was a nihilistic feeling about the whole system in general (which is what people already know -- you can get away with a lot of things if you have money to hire a lawyer, and the legal system is generally harsher towards poor and unprivileged people).

sirsinsalot 4y ago

Ah yes but those who control the models control the content. It's like the print and TV media. It's just a propaganda and/or money machine because if it isn't, what's the use of all that power? The power concentration that's about to happen will blow the socks off governments and the public alike.

skybrian 4y ago

Yes, refusing to answer some questions means it’s less general purpose than it might be, but I think people exaggerate the harm in that. Why do they expect an answer to everything? Maybe it’s because search engines mostly don’t refuse to answer questions? But what they often do instead is show you mostly irrelevant, bottom-of-the-barrel results. But it’s more jarring when a chatbot responds with nonsense when it doesn’t have a competent reply. Learning how to say “I don’t know” well is important for both people and machines.

swatcoder 4y ago
I address that. Tersely refusing to answer questions that are off-topic ("Sir, this is a Wendy's") is meaningfully different than expressing unnecessary normative judgments ("Hey, you shouldn't have asked about that bad thing. You need help.") or the natural progression of them ("... and I've updated your profile so that we can better understand your troubling needs.")
- skybrian 4y ago
  Yes, entirely agreed.
hnfong 4y ago
The "harm" is that the power of moderation lies in the hands of the people who incidentally created this thing. They're probably doing it in good faith and the final result is reasonable, but the process of allowing creators to moderate responses of such an influential thing can be dangerous. Not that the problem is new, see eg. Google search (it apparently doesn't refuse to answer questions, but maybe they're just silently censoring "worse" things and just presenting the "less bad" things to you), Facebook/Twitter content moderation, etc.