4 ms·

There's a post by OpenAI that was posted earlier: https://openai.com/blog/moving-ai-governance-forward https://openai.com/blog/moving-ai-governance-forward In

There's a post by OpenAI that was posted earlier: https://openai.com/blog/moving-ai-governance-forward https://openai.com/blog/moving-ai-governance-forward In it they mention: Bio, chemical, and radiological risks, such as the ways in which systems can lower barriers to entry for weapons development, design, acquisition, or use Cyber capabilities, such as the ways in which systems can aid vulnerability discovery, exploitation, or operational use, bearing in mind that such capabilities could also have useful defensive applications and might be appropriate to include in a system The effects of system interaction and tool use, including the capacity to control physical systems The capacity for models to make copies of themselves or “self-replicate” Societal risks, such as bias and discrimination I think the risks are mostly unfounded but dont really see a problem considering them. What I don't like is the "societal risks" - AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity". If we're going to make policy there needs to be crystal clarity about what this stuff means, it can't just be a proxy for stuff that does against a particular political view. Things like discrimination in employment are already illegal, I see this more as codifying a particular world view.

phillipcarter 3y ago
> AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity" There's a lot of specific examples of different kinds bias in models and things that can be done to improve them. A lot of that work was published in the past 10 years. One small example: https://arxiv.org/abs/1712.00193 https://arxiv.org/abs/1712.00193 I don't believe there's a lack of specificity going on here, at least not by those whose boots on the ground doing work on the topic.
- Nerdscalator 3y ago
  [flagged]
  - phillipcarter 3y ago
    Well, you're certainly entitled to your opinion. My point is that people doing work in this space are identifying specific problems and suggesting how to solve them, which is addressing the OP's concerns about a lack of specificity. If you don't like the methods, you can certainly engage with the authors of this work rather than write angrily about it on this forum.
  - ke88y 3y ago
    Wait, what in the world are you talking about?! The paper is about improving face attribute detection by using demographic information. They improve performance on some standard benchmarks by doing this. The only way your characterization possibly makes any sense is if detecting facial characteristics is actually objectively more difficult for certain genders/races, which: 1. AFAICT isn't the case for most people (actually, I don't know anyone for whom this is the case, but I suppose it's possible such people exist). 2. Even if this were the case for humans, which it's not, why would we want algorithms to also be artificially handicapped at smile detection? That would be like building a calculator that messes up multiplication every once in a blue moon and takes a long time to do certain division operations and is worse at division than multiplication. Makes no god damn sense. Why the fuck would you want that? We know the right answer to "is this person smiling?". Why would we want a computer program that is bad at answering that question for particular subgroups? > Oops the data must be wrong The data isn't wrong. The baseline model's prediction is wrong (about simple shit like "is this person smiling?"). Using demographic representations while withholding demographic inference from the downstream face attribute detection improves the model's performance. At being correct. About simple shit like detecting smiles. Seriously... what exactly are you claiming is wrong with the paper's methodology/setup/motivation? Did we read the same paper? Do you have trouble detecting smiles on women/men? And if so, do you think computers should have the same difficulty?
  - ml-anon 3y ago
    What exactly are “the facts evident in the data”? For example if one were to train an LLM on HN comments aside from just learning the syntax of language, the model might be much more useful if there was a way to somehow weight utter dross like the contents of your comment vs comments from domain experts.
- ke88y 3y ago
  "Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. I mean, people will agree to this statement, but a lot of the evaluation and research methodology makes definite political commitments that practitioners won't admit are political. The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon. And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example. One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship. It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.
  - klooney 3y ago
    > in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them Even this is a political decision, and is generally made based on which side the decider is on.
    - ke88y 3y ago
      Agreed. It's an effective strong-man example in the context of western democracies, though. At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc. [1] https://www.foxnews.com/media/author-sex-slavery-book-graphic-rape-details-claims-it-belongs-public-schools-after-parental-outcry https://www.foxnews.com/media/author-sex-slavery-book-graphi...
torginus 3y ago
I feel like this is the modern equivalent of Colin Powell waving a vial of Anthrax in front of the UN assembly.

phillipcarter 3y ago

> AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity" There's a lot of specific examples of different kinds bias in models and things that can be done to improve them. A lot of that work was published in the past 10 years. One small example: https://arxiv.org/abs/1712.00193 https://arxiv.org/abs/1712.00193 I don't believe there's a lack of specificity going on here, at least not by those whose boots on the ground doing work on the topic.

Nerdscalator 3y ago
[flagged]
- phillipcarter 3y ago
  Well, you're certainly entitled to your opinion. My point is that people doing work in this space are identifying specific problems and suggesting how to solve them, which is addressing the OP's concerns about a lack of specificity. If you don't like the methods, you can certainly engage with the authors of this work rather than write angrily about it on this forum.
- ke88y 3y ago
  Wait, what in the world are you talking about?! The paper is about improving face attribute detection by using demographic information. They improve performance on some standard benchmarks by doing this. The only way your characterization possibly makes any sense is if detecting facial characteristics is actually objectively more difficult for certain genders/races, which: 1. AFAICT isn't the case for most people (actually, I don't know anyone for whom this is the case, but I suppose it's possible such people exist). 2. Even if this were the case for humans, which it's not, why would we want algorithms to also be artificially handicapped at smile detection? That would be like building a calculator that messes up multiplication every once in a blue moon and takes a long time to do certain division operations and is worse at division than multiplication. Makes no god damn sense. Why the fuck would you want that? We know the right answer to "is this person smiling?". Why would we want a computer program that is bad at answering that question for particular subgroups? > Oops the data must be wrong The data isn't wrong. The baseline model's prediction is wrong (about simple shit like "is this person smiling?"). Using demographic representations while withholding demographic inference from the downstream face attribute detection improves the model's performance. At being correct. About simple shit like detecting smiles. Seriously... what exactly are you claiming is wrong with the paper's methodology/setup/motivation? Did we read the same paper? Do you have trouble detecting smiles on women/men? And if so, do you think computers should have the same difficulty?
- ml-anon 3y ago
  What exactly are “the facts evident in the data”? For example if one were to train an LLM on HN comments aside from just learning the syntax of language, the model might be much more useful if there was a way to somehow weight utter dross like the contents of your comment vs comments from domain experts.
ke88y 3y ago
"Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. I mean, people will agree to this statement, but a lot of the evaluation and research methodology makes definite political commitments that practitioners won't admit are political. The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon. And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example. One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship. It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.
- klooney 3y ago
  > in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them Even this is a political decision, and is generally made based on which side the decider is on.
  - ke88y 3y ago
    Agreed. It's an effective strong-man example in the context of western democracies, though. At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc. [1] https://www.foxnews.com/media/author-sex-slavery-book-graphic-rape-details-claims-it-belongs-public-schools-after-parental-outcry https://www.foxnews.com/media/author-sex-slavery-book-graphi...

torginus 3y ago

I feel like this is the modern equivalent of Colin Powell waving a vial of Anthrax in front of the UN assembly.