4 ms·

I assume you are referring to this section: > Create or sell an AI system or product that has “an adverse or disproportionate impact on members of a protected

I assume you are referring to this section: > Create or sell an AI system or product that has “an adverse or disproportionate impact on members of a protected class, or create, reinforce, or perpetuate discrimination or segregation of members of a protected class.“ It's a straw man to characterize that as saying disparate outcomes ALONE are why AI systems might be discriminatory. They may well can be, and likely are, actually embedding biases in the models. The simple examples of how many systems often prefer "he" for doctors and "her" for nurses shows that bias in datasets results in bias in the models. Yes, that is a result of the dataset and reflects the dataset (and maybe even real world statistics on doctors and nurses!) but it does mean the system may treat women and men differently when there is no legal justification to do so.

intalentive 2y ago
I reject the notion that legal justification is necessary. Your LLM prefers "he" for doctors and "her" for nurses, and therefore you can be prosecuted by the state? That's ridiculous. All such Harrison Bergeron-style speech-policing is antithetical to a free society.
- advisedwang 2y ago
  No, I'm not saying the he/she thing is worthy of prosecution. Straw man #2, nice. It is just an undeniable illustration that bias is encoded in machine learning models. Of course it requires some actual harm before the law is involved. Perhaps AI recruiting systems favour resumes that match the "correct" gender to profession. Perhaps a hypothetical AI sentencing recommender gives stiffer penalties to people that live in high crime areas; after all, statistically they are more likely to re-offend - that's just not an acceptable sentencing factor. Perhaps a house appraisal AI accidentally recreates redlining. These are just hard to prove, so my example is there to demonstrate that even though it's hard to demonstrate we should take the risk seriously.
  - intalentive 2y ago
    >or create, reinforce, or perpetuate discrimination Perpetuating "harmful stereotypes" reasonably falls under this language. Trying to correct the bias of the dataset, or worse, correcting the bias of reality, is a fool's errand. The same motive produced the Google AI images controversy last year and constitutes a substantial percentage of instruction fine-tuning for "alignment". I agree that care should be exercised in rolling out algorithms for sentencing guidelines.

intalentive 2y ago

I reject the notion that legal justification is necessary. Your LLM prefers "he" for doctors and "her" for nurses, and therefore you can be prosecuted by the state? That's ridiculous. All such Harrison Bergeron-style speech-policing is antithetical to a free society.

advisedwang 2y ago
No, I'm not saying the he/she thing is worthy of prosecution. Straw man #2, nice. It is just an undeniable illustration that bias is encoded in machine learning models. Of course it requires some actual harm before the law is involved. Perhaps AI recruiting systems favour resumes that match the "correct" gender to profession. Perhaps a hypothetical AI sentencing recommender gives stiffer penalties to people that live in high crime areas; after all, statistically they are more likely to re-offend - that's just not an acceptable sentencing factor. Perhaps a house appraisal AI accidentally recreates redlining. These are just hard to prove, so my example is there to demonstrate that even though it's hard to demonstrate we should take the risk seriously.
- intalentive 2y ago
  >or create, reinforce, or perpetuate discrimination Perpetuating "harmful stereotypes" reasonably falls under this language. Trying to correct the bias of the dataset, or worse, correcting the bias of reality, is a fool's errand. The same motive produced the Google AI images controversy last year and constitutes a substantial percentage of instruction fine-tuning for "alignment". I agree that care should be exercised in rolling out algorithms for sentencing guidelines.