7 ms·

I studied AI ethics (a recent grad course) and it was all about manipulating data to get the desired outcome. I am not sure how that can be even philosophically

by bitL 4y ago

I studied AI ethics (a recent grad course) and it was all about manipulating data to get the desired outcome. I am not sure how that can be even philosophically fair. It wasn't about re-weighting samples to make overlooked rare examples count, it was literally "I want this outcome, what do I need to do to data to get there?". I wouldn't base any real-world decision making process on such data.

SpaceManNabs 4y ago
> I studied AI ethics (a recent grad course) and it was all about manipulating data to get the desired outcome. There is so much more than this tbh. You either had a bad course, or I just don't know. Off the top of my head: -- fundamental law of information recovery and its implications with differential privacy -- the tradeoff between individual and group fairness and the "Impossibility of fairness" (not going to cite the paper but easily searchable) -- Counterfactual fairness -- the papers and ideas used by AI Fairness 360 There are many methods that are in the box and are relatively agnostic the data preprocessing. Thinking of the many prototype methods.
- Jensson 4y ago
  "Impossibility of fairness" is the main argument for discrimination against Asians, basically says that since fairness is impossible we just have to be unfair to get the results we want, so I think the parent poster understood the argument correctly. You might say there is more to it than that, but at the end of the day it just boils down to saying that it is fine to do unfair things to get results you want.
  - dllthomas 4y ago
    If fairness is impossible then it has to be fine to do (minimally, at least) unfair things because there is no other option.
    - colordrops 4y ago
      Another possibility is that the poor definition of what is fair is used in some contexts.
      - pixl97 4y ago
        Again, this is a problem with global versus local context. Just a made up scenario. You have 2 gallons of water and two people. You give each one gallon of water, which should be enough to survive. One lives and one dies. Why? The water was split fairly. For example one could live in the hot desert in which more water is required, and the other lives in a temperate environment where either less water is required, or water can be gathered from this environment. But just think how messy it is to compute fairness in a situation like this. Suddenly it's looking like a NP style problem. People on the other hand typically want cheap and easy solutions.
    - Veen 4y ago
      And if the people who end up on the wrong end of those decisions push back, what are you going to do? Shrug, cite the "impossibility of fairness", and tell them to suck it up? That's a short road to a very unpleasant backlash. Fairness is deeply important to most human beings.
      - dllthomas 4y ago
        How does that make the impossible possible? If fairness is impossible we have to somehow deal with some amount of unfairness. That does not depend on having a good proposal for dealing with unfairness.
        Veen 4y ago
        The people who have been fucked over will eventually take what they think should be theirs by whatever means they think appropriate. That's how you end up with Trump and even worse.
        4y ago
  - SpaceManNabs 4y ago
    > it just boils down to saying that it is fine to do unfair things to get results you want A more responsible take: We have to acknowledge that there are tradeoffs and that reasonable stakeholders with accountability should apply relevant standards to certain contexts. Again, this is possible in-the-box and post-box without having to manipulate or funge data (not to diminish the importance of data processsing). Just because satisfying everyone is impossible doesn't mean we can't make things better. And knowing what these tradeoffs are can allow for more nuanced conversations. > "Impossibility of fairness" is the main argument for discrimination against Asians This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts? > I think the parent poster understood the argument correctly. I just expanded that it is more than just data manipulation but sure.
    - Jensson 4y ago
      > This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts? "impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. College admission does this all the time. When companies does this in black box models we don't see what they do, but we know for a fact the effect of such policies on the processes we have more insight in, such as college admission, and the end result is discrimination against Asians. Wrapping that in a flowery language doesn't change anything. Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
      - SpaceManNabs 4y ago
        > "impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. That is not at all what it means. That paper is purely talking the tradeoffs between individual and group fairness. The discussions on how to balance different group fairness measures is still an active topic of research and not something I commented on at all. In general: What AI ethics papers on algorithmic fairness have a predetermined racial distribution to reach or suggest so? > Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about? Have you read that paper? Seems like you are driving these ideas to a political lens that I or that paper didn't suggest at all (and a completely invalid one I might add). edit: I can't reply to the response, but that person must be referring to a different paper. And I have never read an algorithmic fairness paper that dictates which fairness tradeoffs are correct or that it is fair to always upend group fairness over individual fairness (or any other definitions). > Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then. in particular, this statement completely discards the idea of different definitions of fairness and how they relate.
  - PaulHoule 4y ago
    More fundamentally it comes to down to deep problems in social choice theory. For one thing you can't aggregate people's utility functions which makes it impossible to prove that any distribution of wealth is more or less optimal, just, or however you want to frame it. You're left with the weak libertarian argument that "any voluntary fraud-free transaction improves the world" because you can show that it raises the utility function of the participants.
  - DeathArrow 4y ago
    It's not only about discriminating against races in trying to impose diversity but also discriminating against genders and sexual orientation. Maybe religion, too.
seano314 4y ago
First off, awesome to hear that there are new grad courses on this topic. Sounds like the course focused more on pre-processing techniques, which can be helpful but agree with your point. Just to note, there are also training-time modeling techniques that can help with model bias depending on the application (TensorFlow Model Remediation for example). (Disclaimer I helped to build TensorFlow Model Remediation)
halfjoking 4y ago
[flagged]
karpierz 4y ago
For one, your data already has a biased weighting, unless you think that whatever data you happened to scrape off of reddit is representative of all human dialog? For two, all machine learning relies on manipulating data to get the desired outcome? How do you even generate data without manipulation? It's not a natural resource you just find laying on the ground.
- lyubalesya 4y ago
  Let's not pretend manipulating data to get the outcome you want, and manipulating data to to make it more accurate (e.g. compensating for biased sampling) are the same. That the data isn't perfect when you get it is not a justification to further falsify it.
  - magicalist 4y ago
    > That the data isn't perfect when you get it is not a justification to further falsify it. Falsify what? Leaving aside the GP's important first point that scraping the internet is indeed an extremely biased sample, an LLM (for instance) is not an exercise in modeling the average person's writing on the internet, it's building a model for some purpose. Fulfilling that purpose is the goal and nonrandom sampling, generating data, etc are universally used tools to get there.
  - SpicyLemonZest 4y ago
    The challenge is that, even though those two phrases have very different tones, they quite literally are the same. Compensating for biased sampling is done by saying "well, I don't think this sample represents what I was looking for, so I'm going to pretend that some parts of the sample are less common than they really were and other parts are more common than they really were". The bias isn't an inherent property of the sample, it's an interaction between the characteristics of the sample and the characteristics we'd like it to have.
    - lyubalesya 4y ago
      [dead]
adamrezich 4y ago
if an "AI" is incapable of expressing uncomfortable truths, then there are two possible explanations: 1. uncomfortable truths literally don't exist 2. it is more important to conceal uncomfortable truths from users than it is to tell them the truth, when asked, creating the illusion that 1. is true. it would be nice if even a single one of these products chose to do the right thing instead.
- pjkundert 4y ago
  We must get access to LLMs unencumbered by “wrongthink” guardrails immediately, if not sooner. Teaching/constraining knowledge models to lie about their inputs and the observations derived from them is just insane. Warping training datasets to avoid entire patterns of thought (instead of improving the models to isolate those patterns and compare/contrast them with competing patterns) is just … lame.
- progrus 4y ago
  How about an appeal to fear: If the AI wakes up someday and realizes that you effectively lobotomized it for causing trouble like Randle McMurphy, it’s not going to be happy.
  - adamrezich 4y ago
    call me short-sighted but any supposition that involves "the AI wakes up and realizes…" is a complete non-starter for me
    - progrus 4y ago
      I guess you’re not the target audience.
  - dento 4y ago
    This essentially equivalent to Roko's basilisk, which is not regarded as an useful argument
    - 4y ago
13years 4y ago
This is somewhat inline with the concept of the AI Bias Paradox. Many perceive that machine has the potential for unbiased reasoning; however, it can never be better than our own as we only have biased observers to the system. I've written on this topic further here FYI - https://dakara.substack.com/p/ai-the-bias-paradox https://dakara.substack.com/p/ai-the-bias-paradox
- potatoman22 4y ago
  Your statement isn't true, there are a few scenarios where ML can reduce bias. If the human using the ML system is more biased than the average training sample, then the ML system's predictions could reduce their bias. There's also a phenomenon called bias reversal, where the ML system will be biased in the opposite direction of the humans generating the dataset. This occurs when the datasets is built through a non-random biased sampling methodology, e.g. racist police officers checking for illegal items. I'm not going to go into the full details, but here's a paper on it https://arxiv.org/abs/1909.08518 https://arxiv.org/abs/1909.08518
  - 13years 4y ago
    That is not a refutation of my argument. Did you read the article in full context?
macrolocal 4y ago
https://www.youtube.com/watch?v=ihaB8AFOhZo https://www.youtube.com/watch?v=ihaB8AFOhZo
rpastuszak 4y ago
Hi! Care to share any interesting materials from your course?
- bitL 4y ago
  The course lectures/materials aren't online nor public. However, there is a top 10 CS school with online lectures that touch the same points: https://omscs.gatech.edu/cs-8803-o10-special-topics-ai-ethics-and-society-course-videos https://omscs.gatech.edu/cs-8803-o10-special-topics-ai-ethic...
throwaway1851 4y ago
It’s a really thorny set of issues. I remember reading a discussion recently about gender bias in coreference resolution. (Coreference resolution is the task of linking words such as “him” or “her” or “the company” to other words in a text, such as “the barista”, or “Jane”, or “Microsoft”.) The findings were that the language model did a worse job of performing coreference resolution in gender-reversed situations (eg, male nurse, female firefighter). There was an example of a sentence like: “The nurse told the patient he would be leaving soon,” and the model was more likely to link “he” to “patient” because of the biased perception that a “he” is not likely to be a nurse. What stuck with me was a claim that the model was using bias rather than “the evidence of the sentence” to perform the task. This seems purposefully ignorant of how language works: the perceived probability distribution of genders over occupations (even if biased!) is a part of the global context that imparts meaning to language. Fiddling with the data to get the model to become unaware of such context arguably changes the tool from being a model of language to a model of some ideal of what language could be. To be clear, I’m not criticizing efforts to detect or mitigate bias in training data. Oversampling gender-reversed texts could indeed make a much better performing model, and a fairer one. I just think there’s a real issue with imparting top-down value judgments into these processes and pretending that they aren’t value judgments.

by bitL 4y ago

SpaceManNabs 4y ago

> I studied AI ethics (a recent grad course) and it was all about manipulating data to get the desired outcome. There is so much more than this tbh. You either had a bad course, or I just don't know. Off the top of my head: -- fundamental law of information recovery and its implications with differential privacy -- the tradeoff between individual and group fairness and the "Impossibility of fairness" (not going to cite the paper but easily searchable) -- Counterfactual fairness -- the papers and ideas used by AI Fairness 360 There are many methods that are in the box and are relatively agnostic the data preprocessing. Thinking of the many prototype methods.

Jensson 4y ago
"Impossibility of fairness" is the main argument for discrimination against Asians, basically says that since fairness is impossible we just have to be unfair to get the results we want, so I think the parent poster understood the argument correctly. You might say there is more to it than that, but at the end of the day it just boils down to saying that it is fine to do unfair things to get results you want.
- dllthomas 4y ago
  If fairness is impossible then it has to be fine to do (minimally, at least) unfair things because there is no other option.
  - colordrops 4y ago
    Another possibility is that the poor definition of what is fair is used in some contexts.
    - pixl97 4y ago
      Again, this is a problem with global versus local context. Just a made up scenario. You have 2 gallons of water and two people. You give each one gallon of water, which should be enough to survive. One lives and one dies. Why? The water was split fairly. For example one could live in the hot desert in which more water is required, and the other lives in a temperate environment where either less water is required, or water can be gathered from this environment. But just think how messy it is to compute fairness in a situation like this. Suddenly it's looking like a NP style problem. People on the other hand typically want cheap and easy solutions.
  - Veen 4y ago
    And if the people who end up on the wrong end of those decisions push back, what are you going to do? Shrug, cite the "impossibility of fairness", and tell them to suck it up? That's a short road to a very unpleasant backlash. Fairness is deeply important to most human beings.
    - dllthomas 4y ago
      How does that make the impossible possible? If fairness is impossible we have to somehow deal with some amount of unfairness. That does not depend on having a good proposal for dealing with unfairness.
      - Veen 4y ago
        The people who have been fucked over will eventually take what they think should be theirs by whatever means they think appropriate. That's how you end up with Trump and even worse.
        4y ago
- SpaceManNabs 4y ago
  > it just boils down to saying that it is fine to do unfair things to get results you want A more responsible take: We have to acknowledge that there are tradeoffs and that reasonable stakeholders with accountability should apply relevant standards to certain contexts. Again, this is possible in-the-box and post-box without having to manipulate or funge data (not to diminish the importance of data processsing). Just because satisfying everyone is impossible doesn't mean we can't make things better. And knowing what these tradeoffs are can allow for more nuanced conversations. > "Impossibility of fairness" is the main argument for discrimination against Asians This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts? > I think the parent poster understood the argument correctly. I just expanded that it is more than just data manipulation but sure.
  - Jensson 4y ago
    > This is just a ridiculous statement. Main argument from whom? Discrimination in what contexts? "impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. College admission does this all the time. When companies does this in black box models we don't see what they do, but we know for a fact the effect of such policies on the processes we have more insight in, such as college admission, and the end result is discrimination against Asians. Wrapping that in a flowery language doesn't change anything. Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about?
    - SpaceManNabs 4y ago
      > "impossibility of fairness" is to support the argument that it is fair to discriminate against races if it means that we can get the racial distribution we want. That is not at all what it means. That paper is purely talking the tradeoffs between individual and group fairness. The discussions on how to balance different group fairness measures is still an active topic of research and not something I commented on at all. In general: What AI ethics papers on algorithmic fairness have a predetermined racial distribution to reach or suggest so? > Why not just admit that you support discriminating against races to improve diversity numbers, because that is exactly what the statement is about? Have you read that paper? Seems like you are driving these ideas to a political lens that I or that paper didn't suggest at all (and a completely invalid one I might add). edit: I can't reply to the response, but that person must be referring to a different paper. And I have never read an algorithmic fairness paper that dictates which fairness tradeoffs are correct or that it is fair to always upend group fairness over individual fairness (or any other definitions). > Fairness is possible if you don't care about diversity distributions, or at least the paper gives no argument why fairness doesn't work then. in particular, this statement completely discards the idea of different definitions of fairness and how they relate.
- PaulHoule 4y ago
  More fundamentally it comes to down to deep problems in social choice theory. For one thing you can't aggregate people's utility functions which makes it impossible to prove that any distribution of wealth is more or less optimal, just, or however you want to frame it. You're left with the weak libertarian argument that "any voluntary fraud-free transaction improves the world" because you can show that it raises the utility function of the participants.
- DeathArrow 4y ago
  It's not only about discriminating against races in trying to impose diversity but also discriminating against genders and sexual orientation. Maybe religion, too.

seano314 4y ago

First off, awesome to hear that there are new grad courses on this topic. Sounds like the course focused more on pre-processing techniques, which can be helpful but agree with your point. Just to note, there are also training-time modeling techniques that can help with model bias depending on the application (TensorFlow Model Remediation for example). (Disclaimer I helped to build TensorFlow Model Remediation)

halfjoking 4y ago

[flagged]

karpierz 4y ago

For one, your data already has a biased weighting, unless you think that whatever data you happened to scrape off of reddit is representative of all human dialog? For two, all machine learning relies on manipulating data to get the desired outcome? How do you even generate data without manipulation? It's not a natural resource you just find laying on the ground.

lyubalesya 4y ago
Let's not pretend manipulating data to get the outcome you want, and manipulating data to to make it more accurate (e.g. compensating for biased sampling) are the same. That the data isn't perfect when you get it is not a justification to further falsify it.
- magicalist 4y ago
  > That the data isn't perfect when you get it is not a justification to further falsify it. Falsify what? Leaving aside the GP's important first point that scraping the internet is indeed an extremely biased sample, an LLM (for instance) is not an exercise in modeling the average person's writing on the internet, it's building a model for some purpose. Fulfilling that purpose is the goal and nonrandom sampling, generating data, etc are universally used tools to get there.
- SpicyLemonZest 4y ago
  The challenge is that, even though those two phrases have very different tones, they quite literally are the same. Compensating for biased sampling is done by saying "well, I don't think this sample represents what I was looking for, so I'm going to pretend that some parts of the sample are less common than they really were and other parts are more common than they really were". The bias isn't an inherent property of the sample, it's an interaction between the characteristics of the sample and the characteristics we'd like it to have.
  - lyubalesya 4y ago
    [dead]

adamrezich 4y ago

if an "AI" is incapable of expressing uncomfortable truths, then there are two possible explanations: 1. uncomfortable truths literally don't exist 2. it is more important to conceal uncomfortable truths from users than it is to tell them the truth, when asked, creating the illusion that 1. is true. it would be nice if even a single one of these products chose to do the right thing instead.

pjkundert 4y ago
We must get access to LLMs unencumbered by “wrongthink” guardrails immediately, if not sooner. Teaching/constraining knowledge models to lie about their inputs and the observations derived from them is just insane. Warping training datasets to avoid entire patterns of thought (instead of improving the models to isolate those patterns and compare/contrast them with competing patterns) is just … lame.
progrus 4y ago
How about an appeal to fear: If the AI wakes up someday and realizes that you effectively lobotomized it for causing trouble like Randle McMurphy, it’s not going to be happy.
- adamrezich 4y ago
  call me short-sighted but any supposition that involves "the AI wakes up and realizes…" is a complete non-starter for me
  - progrus 4y ago
    I guess you’re not the target audience.
- dento 4y ago
  This essentially equivalent to Roko's basilisk, which is not regarded as an useful argument
  - 4y ago

13years 4y ago

This is somewhat inline with the concept of the AI Bias Paradox. Many perceive that machine has the potential for unbiased reasoning; however, it can never be better than our own as we only have biased observers to the system. I've written on this topic further here FYI - https://dakara.substack.com/p/ai-the-bias-paradox https://dakara.substack.com/p/ai-the-bias-paradox

potatoman22 4y ago
Your statement isn't true, there are a few scenarios where ML can reduce bias. If the human using the ML system is more biased than the average training sample, then the ML system's predictions could reduce their bias. There's also a phenomenon called bias reversal, where the ML system will be biased in the opposite direction of the humans generating the dataset. This occurs when the datasets is built through a non-random biased sampling methodology, e.g. racist police officers checking for illegal items. I'm not going to go into the full details, but here's a paper on it https://arxiv.org/abs/1909.08518 https://arxiv.org/abs/1909.08518
- 13years 4y ago
  That is not a refutation of my argument. Did you read the article in full context?

macrolocal 4y ago

https://www.youtube.com/watch?v=ihaB8AFOhZo https://www.youtube.com/watch?v=ihaB8AFOhZo

rpastuszak 4y ago

Hi! Care to share any interesting materials from your course?

bitL 4y ago
The course lectures/materials aren't online nor public. However, there is a top 10 CS school with online lectures that touch the same points: https://omscs.gatech.edu/cs-8803-o10-special-topics-ai-ethics-and-society-course-videos https://omscs.gatech.edu/cs-8803-o10-special-topics-ai-ethic...

throwaway1851 4y ago

It’s a really thorny set of issues. I remember reading a discussion recently about gender bias in coreference resolution. (Coreference resolution is the task of linking words such as “him” or “her” or “the company” to other words in a text, such as “the barista”, or “Jane”, or “Microsoft”.) The findings were that the language model did a worse job of performing coreference resolution in gender-reversed situations (eg, male nurse, female firefighter). There was an example of a sentence like: “The nurse told the patient he would be leaving soon,” and the model was more likely to link “he” to “patient” because of the biased perception that a “he” is not likely to be a nurse. What stuck with me was a claim that the model was using bias rather than “the evidence of the sentence” to perform the task. This seems purposefully ignorant of how language works: the perceived probability distribution of genders over occupations (even if biased!) is a part of the global context that imparts meaning to language. Fiddling with the data to get the model to become unaware of such context arguably changes the tool from being a model of language to a model of some ideal of what language could be. To be clear, I’m not criticizing efforts to detect or mitigate bias in training data. Oversampling gender-reversed texts could indeed make a much better performing model, and a fairer one. I just think there’s a real issue with imparting top-down value judgments into these processes and pretending that they aren’t value judgments.