6 ms·

Specifically with AI I don’t want to experience the downside of innovation before we regulate because of how wide spread its use already is, and it’s problems h

Specifically with AI I don’t want to experience the downside of innovation before we regulate because of how wide spread its use already is, and it’s problems have already become apparent. For example, it’s being used to job screen applicants even though we have proven that AI models still suffer from thing like racial bias. Companies don’t disclose how their models are trained to negate bias or anything like that either and that’s one example I remember off the top of my head

nottorp 2y ago
> it’s being used to job screen applicants even though we have proven that AI models still suffer from thing like racial bias I bet they also suffer from other biases that are harder to detect and maybe some biases we can't even imagine and thus control for.
- deleted 2y ago
  [deleted]
- chfalck 2y ago
  Kind of like humans in that regard
  - petre 2y ago
    Except being able to process 1e9 more applications, thus massively more biased and discriminatory.
- paulddraper 2y ago
  So.....,just like humans?
  - killingtime74 2y ago
    That's the point though. It's illegal for humans to do things during hiring that are not illegal for AI to do as it stands now (all sorts of discrimination). We want to at least level the playing field.
    - simonh 2y ago
      There’s no exemption from prosecution for racial bias because it was done by an algorithm. If a company uses such an algorithm, AI or not, they will be open to prosecution for it.
      - no_wizard 2y ago
        The differential nugget is that LLMs and similar tech are auditable. You can audit the models, test for bias etc in a way that scales.
        vrighter 2y ago
        how do you audit the loaded die at the end of all llms?
        mzmzmzm 2y ago
        Are there agreed upon neutral auditors or common processes for auditing models? If not, maybe what we need is (gasp!) some governmental oversight.
  - mzmzmzm 2y ago
    Humans can be asked to explain their decissions and personally take accountability for them. Even if AI processes were less biased, using them introduces the opacity of relying on models that simply can't be fully intuited or explained. For life-altering decisions, merely performing better than a human is not the benchmark.
627467 2y ago
there maybe better examples of why AI biases provide deep systemic problems but: CV screening? are contemporary LLM really worse that previous screening tech and processes?
- ivan_gammel 2y ago
  You can combat bias in a team of people, but in LLM you may not even know it exists.
  - fallingknife 2y ago
    How can you combat bias in a team of humans if humans are susceptible to bias? How do you even know what neutral is if it's not a physically measurable quantity?
    - Larrikin 2y ago
      This sounds like an argument for never educating people and doing nothing about the problem. Just because you graduated from a CS program doesn't mean you will always write bug free code using the ideal algorithm and design pattern. But you know what you should be striving for, can more readily identify issues, and maybe sometimes you actually are perfect.
      - AnthonyMouse 2y ago
        > This sounds like an argument for never educating people and doing nothing about the problem. I feel like it's really the opposite. There is a major problem with using AI to screen CVs, which is that AI is bad at screening CVs. It excludes good candidates and offers mediocre ones, and opens up a bunch of hacks the equivalent of black hat SEO, which gives an undesired advantage to people inclined to dirty tricks. But "it's racist" is a political hot button, so before people can start to point that out, someone scrambles to push the hot button and suck all the oxygen out of the room. Then the purveyors of snake oil can munge the algorithm until it passes the naive oversimplified racial bias test someone made up (probably making it even worse at its intended purpose), and then claim "the problem" is solved, as if that was the main and only problem.
Silhouette 2y ago
I would argue that we already have experienced enough of the downsides of "AI" that there is reasonable cause for concern. The implications of deepfakes and similar frauds alone are potentially devastating to informed political debate in democracies, safe and effective dissemination of public health information in emergencies, and plenty of other realistic and important trust scenarios. The implications of LLMs are potentially wonderful in terms of providing better access to information for everyone but we already know that they are also capable of making serious mistakes or even generating complete nonsense that a non-expert user might not recognise as such. Again it is not hard to imagine a near future where chat-based systems have essentially displaced search engines and social media as the default ways to find information online but then provide bad advice on legal, financial, or health matters. There is a second serious concern with LLMs and related technologies, which is that they could very rapidly shift the balance from compensating those who produce useful creative content to compensating those who run the summary service. It's never healthy when your economics don't line up with rewarding the people doing the real work and we've already seen plenty of relevant stories about the AI training data gold rush. Next we get to computer vision and its applications in fields like self-driving vehicles. Again we've already seen plenty of examples where cars have been tricked into stopping suddenly or otherwise misbehaving when for example someone projected a fake road sign onto the road in front of them. Again there is a second serious concern with systems like computer vision, audio classification, and natural language processing and that is privacy. It's bad enough that we all carry devices with cameras and microphones around with us almost 24/7 these days and the people whose software runs on those devices seem quite willing to spy on us and upload data to the mothership with little or any warning. That alone has unprecedented implications for privacy and associated risks. With the increased ability to automatically interpret raw video and audio footage - with varying degrees of accuracy and bias of course - that amplifies the potential dangers of these systems greatly. There is enormous potential in modern AI/ML techniques for everything from helping everyday personal research to saving lives through commoditising sophisticated analysis of medical scans. But that doesn't mean there aren't also risks we already know about at the same kind of scale - even without all the doomsday hypotheticals where suddenly a malicious AGI emerges that takes over the universe.
- AnthonyMouse 2y ago
DelightOne 2y ago
Problem is most innovation is in using existing models in new ways. You can't expect these most people to train their own models. Regulating it the way you say just means "zero innovation!".
inquirerGeneral 2y ago
[dead]
bawolff 2y ago
> For example, it’s being used to job screen applicants even though we have proven that AI models still suffer from thing like racial bias. Can't we just say racism is illegal, and if a company uses an AI to be racist, they get fined the same way they would if they were racist the old fashioned way?
- graftak 2y ago
  Look up the Dutch tax return scandal where the Dutch tax arm of the government (‘IRS’) used machine learning to identify fraud but it turned out to be very racially biased and it uprooted thousands of families with years of financial struggles and legal battles. See https://en.m.wikipedia.org/wiki/Dutch_childcare_benefits_scandal https://en.m.wikipedia.org/wiki/Dutch_childcare_benefits_sca...
  - taskforcegemini 2y ago
    or the royal mail horizon scandal. although the software not called an "ai" afair
- ben_w 2y ago
  "Just", no. Fence at the top of a cliff (make sure the AI is unbiased and can be fixed when it turns out it is) vs. Ambulance at the bottom (letting people sue if they think the machine is wrong).
  - bawolff 2y ago
    When you make specific methods illegal, it tends to lead to loopholes. If you just make the result illegal then nobody can try and get around it by using a slightly different system design.
    - ben_w 2y ago
      Also true. But which way around does that apply? Racism is a concrete example of what AI may incorrectly automate, it's not the only bias that AI can have — any correlation in the training data will be learned, but not necessarily causation — and the goal of these laws is to require the machines to be accurate, unbiased, up to date, respectful of privacy etc., with "not racist" as merely one example of why that matters. (Also the existing laws on racial equality were not removed by GDPR; to the previous metaphor, the fence being better doesn't mean you can fire the ambulance service).
apwell23 2y ago
> it’s being used to job screen applicants Any idea what software is being used ?
- petre 2y ago
  Just an example: https://hirebee.ai/ https://hirebee.ai/
paulddraper 2y ago
How do you know the race? I've seen some places collect that information which is wild. But you can decline.

nottorp 2y ago

> it’s being used to job screen applicants even though we have proven that AI models still suffer from thing like racial bias I bet they also suffer from other biases that are harder to detect and maybe some biases we can't even imagine and thus control for.

deleted 2y ago
[deleted]
chfalck 2y ago
Kind of like humans in that regard
- petre 2y ago
  Except being able to process 1e9 more applications, thus massively more biased and discriminatory.
paulddraper 2y ago
So.....,just like humans?
- killingtime74 2y ago
  That's the point though. It's illegal for humans to do things during hiring that are not illegal for AI to do as it stands now (all sorts of discrimination). We want to at least level the playing field.
  - simonh 2y ago
    There’s no exemption from prosecution for racial bias because it was done by an algorithm. If a company uses such an algorithm, AI or not, they will be open to prosecution for it.
    - no_wizard 2y ago
      The differential nugget is that LLMs and similar tech are auditable. You can audit the models, test for bias etc in a way that scales.
      - vrighter 2y ago
        how do you audit the loaded die at the end of all llms?
      - mzmzmzm 2y ago
        Are there agreed upon neutral auditors or common processes for auditing models? If not, maybe what we need is (gasp!) some governmental oversight.
- mzmzmzm 2y ago
  Humans can be asked to explain their decissions and personally take accountability for them. Even if AI processes were less biased, using them introduces the opacity of relying on models that simply can't be fully intuited or explained. For life-altering decisions, merely performing better than a human is not the benchmark.

627467 2y ago

there maybe better examples of why AI biases provide deep systemic problems but: CV screening? are contemporary LLM really worse that previous screening tech and processes?

ivan_gammel 2y ago
You can combat bias in a team of people, but in LLM you may not even know it exists.
- fallingknife 2y ago
  How can you combat bias in a team of humans if humans are susceptible to bias? How do you even know what neutral is if it's not a physically measurable quantity?
  - Larrikin 2y ago
    This sounds like an argument for never educating people and doing nothing about the problem. Just because you graduated from a CS program doesn't mean you will always write bug free code using the ideal algorithm and design pattern. But you know what you should be striving for, can more readily identify issues, and maybe sometimes you actually are perfect.
    - AnthonyMouse 2y ago
      > This sounds like an argument for never educating people and doing nothing about the problem. I feel like it's really the opposite. There is a major problem with using AI to screen CVs, which is that AI is bad at screening CVs. It excludes good candidates and offers mediocre ones, and opens up a bunch of hacks the equivalent of black hat SEO, which gives an undesired advantage to people inclined to dirty tricks. But "it's racist" is a political hot button, so before people can start to point that out, someone scrambles to push the hot button and suck all the oxygen out of the room. Then the purveyors of snake oil can munge the algorithm until it passes the naive oversimplified racial bias test someone made up (probably making it even worse at its intended purpose), and then claim "the problem" is solved, as if that was the main and only problem.

Silhouette 2y ago

I would argue that we already have experienced enough of the downsides of "AI" that there is reasonable cause for concern. The implications of deepfakes and similar frauds alone are potentially devastating to informed political debate in democracies, safe and effective dissemination of public health information in emergencies, and plenty of other realistic and important trust scenarios. The implications of LLMs are potentially wonderful in terms of providing better access to information for everyone but we already know that they are also capable of making serious mistakes or even generating complete nonsense that a non-expert user might not recognise as such. Again it is not hard to imagine a near future where chat-based systems have essentially displaced search engines and social media as the default ways to find information online but then provide bad advice on legal, financial, or health matters. There is a second serious concern with LLMs and related technologies, which is that they could very rapidly shift the balance from compensating those who produce useful creative content to compensating those who run the summary service. It's never healthy when your economics don't line up with rewarding the people doing the real work and we've already seen plenty of relevant stories about the AI training data gold rush. Next we get to computer vision and its applications in fields like self-driving vehicles. Again we've already seen plenty of examples where cars have been tricked into stopping suddenly or otherwise misbehaving when for example someone projected a fake road sign onto the road in front of them. Again there is a second serious concern with systems like computer vision, audio classification, and natural language processing and that is privacy. It's bad enough that we all carry devices with cameras and microphones around with us almost 24/7 these days and the people whose software runs on those devices seem quite willing to spy on us and upload data to the mothership with little or any warning. That alone has unprecedented implications for privacy and associated risks. With the increased ability to automatically interpret raw video and audio footage - with varying degrees of accuracy and bias of course - that amplifies the potential dangers of these systems greatly. There is enormous potential in modern AI/ML techniques for everything from helping everyday personal research to saving lives through commoditising sophisticated analysis of medical scans. But that doesn't mean there aren't also risks we already know about at the same kind of scale - even without all the doomsday hypotheticals where suddenly a malicious AGI emerges that takes over the universe.

AnthonyMouse 2y ago

DelightOne 2y ago

Problem is most innovation is in using existing models in new ways. You can't expect these most people to train their own models. Regulating it the way you say just means "zero innovation!".

inquirerGeneral 2y ago

[dead]

bawolff 2y ago

> For example, it’s being used to job screen applicants even though we have proven that AI models still suffer from thing like racial bias. Can't we just say racism is illegal, and if a company uses an AI to be racist, they get fined the same way they would if they were racist the old fashioned way?

graftak 2y ago
Look up the Dutch tax return scandal where the Dutch tax arm of the government (‘IRS’) used machine learning to identify fraud but it turned out to be very racially biased and it uprooted thousands of families with years of financial struggles and legal battles. See https://en.m.wikipedia.org/wiki/Dutch_childcare_benefits_scandal https://en.m.wikipedia.org/wiki/Dutch_childcare_benefits_sca...
- taskforcegemini 2y ago
  or the royal mail horizon scandal. although the software not called an "ai" afair
ben_w 2y ago
"Just", no. Fence at the top of a cliff (make sure the AI is unbiased and can be fixed when it turns out it is) vs. Ambulance at the bottom (letting people sue if they think the machine is wrong).
- bawolff 2y ago
  When you make specific methods illegal, it tends to lead to loopholes. If you just make the result illegal then nobody can try and get around it by using a slightly different system design.
  - ben_w 2y ago
    Also true. But which way around does that apply? Racism is a concrete example of what AI may incorrectly automate, it's not the only bias that AI can have — any correlation in the training data will be learned, but not necessarily causation — and the goal of these laws is to require the machines to be accurate, unbiased, up to date, respectful of privacy etc., with "not racist" as merely one example of why that matters. (Also the existing laws on racial equality were not removed by GDPR; to the previous metaphor, the fence being better doesn't mean you can fire the ambulance service).

apwell23 2y ago

> it’s being used to job screen applicants Any idea what software is being used ?

petre 2y ago
Just an example: https://hirebee.ai/ https://hirebee.ai/

paulddraper 2y ago

How do you know the race? I've seen some places collect that information which is wild. But you can decline.