4 ms·
What is being argued in the article that AI Companies could be doing illegally: -Using AI to “foster or advance deception.” -Falsely advertising “the accuracy
by htk 2y ago
What is being argued in the article that AI Companies could be doing illegally:
-Using AI to “foster or advance deception.”
-Falsely advertising “the accuracy, quality, or utility of AI systems.”
-Create or sell an AI system or product that has “an adverse or disproportionate impact on members of a protected class, or create, reinforce, or perpetuate discrimination or segregation of members of a protected class.“
- hermannj314 2y agoWhat kind of standard is disproportionate impact? Do not build an AI system that discovers the best way to target men to get a prostate exam, or the elderly to enroll in a program that may benefit them? I understand "adverse", but disproportionate implies don't even try to help classes of people. AT ALL!
- foxyv 2y agoWhat they are talking about is using AI as a proxy for racial, gender, and age discrimination. For instance, if you use AI for an adverse decision regarding employment, housing, education, or financial decisions you need to justify those decisions against persons from a protected class. If you cannot explain why you have made such decisions, then the state will look at disproportionate impact. EG: You hired/lent to/rented to black people 25% less often than you did to white people. "Because the AI said so." doesn't cut it.
- hermannj314 2y agoYour argument makes it sound like the statement says "adverse and disproportionate". I'm seeing that it says OR, meaning you don't have to do anything adverse. I don't like when laws are worded such that we told we shouldn't care what the words say, we all know what we meant, this is a law for getting the bad guys, so don't worry about the actual words we use. You shouldn't need a law degree to know what OR means.
- deleted 2y ago[deleted]
- throwup238 2y agoDon’t make the mistake of confusing colloquial language with legal text. They do need a law degree, or at least knowledge of the precedent that precisely defined the term. Precisely defining them is half the battle. “Adverse or disproportionate impact” is a well litigated phrase. It has a specific meaning in law, which is not immediately obvious from a layman’s definition.
- kenjackson 2y agoLegal writing brings in a lot of context. In this case the disproportionate is meant to denote situations where the action itself isn't adverse, e.g., giving someone money, but that its not applied evenly. Whereas adverse is an intended negative consequence. Its trying to say both are problematic. Although, I understand your interpretation -- its just not what's meant in this case. Unfortunately even arguably our most important legal document, the Constitution, isn't nearly as clear as it could be.
- deleted 2y ago[deleted]
- foxyv 2y agoTypically wording will either be defined in the law, or decided by case law. Legislatures will usually include very long sections on definitions to try and reduce the amount of judicial leeway. For instance, in Texas motor vehicle code regarding driving under the influence, there is a definition for "Motor Vehicle." While colloquially a person would assume that such a vehicle should have a motor, the definition actually states: > "Motor vehicle" means a device in, on, or by which a person or property is or may be transported or drawn on a highway, except a device used exclusively on stationary rails or tracks. When it comes to laws, you have to read the entire law (including definitions) and not just rely on your own understanding of the terms within. Then it gets even more complicated when it comes to so called "Case Law." This is why companies have entire sections of lawyers to inform their managers on compliance.
- hellojesus 2y agoI interviewed for a consulting role circa 2019 where the company audited models of finanical companies that used their models to make lending decisions. A large portion of the interview touched statistical inference as it related to ML, specifically how it related to simple neural nets up to deep learning vs classical modeling. The answer I gave aligned with their expectations, which was that models used in lending should not be black box and should be able to quantify which features led to the prediction/output and how much weight they contributed. This was specifically done to address potential discrimination lawsuits. I have a hard time believing any company that rents, lends, etc. would employ a black box for decisioning. Both private and public lawyers would sue them into oblivion immediately.
- slt2021 2y agothere is an entire specialty of Model Risk Management, banks have entire departments that oversee models used for decision making and ensure compliance with fair lending laws and other laws https://www.occ.treas.gov/publications-and-resources/publications/comptrollers-handbook/files/model-risk-management/pub-ch-model-risk.pdf https://www.occ.treas.gov/publications-and-resources/publica...
- uoaei 2y agoOne prominent counterexample is that made famous after the UnitedHealthcare CEO was shot and killed, that they employed a decision-making algorithm that apparently rejected 90% of claims with no audit trail or real justification besides cost estimates.
- lotsofpulp 2y agoThat is an unsubstantiated (so far) claim by plaintiffs in an active lawsuit. I would not consider it data for any example yet. https://www.theguardian.com/us-news/2025/jan/25/health-insurers-ai https://www.theguardian.com/us-news/2025/jan/25/health-insur...
- Terr_ 2y ago> What they are talking about is using AI as a proxy for racial, gender, and age discrimination. See also: "It's not a crime if you do it with an app" [/s] - https://pluralistic.net/2025/01/25/potatotrac/ https://pluralistic.net/2025/01/25/potatotrac/
- devoutsalsa 2y agoI do work in recruiting. If you let AI making hiring decisions for you, you are an idiot. All it takes is one disgruntled applicant to complain about discrimination in your hiring process to cause you a world of pain. You need to be capable of owning your hiring decisions, meaning you can’t just point to the AI and say “it told me not to hire this person”. You won’t be able to avoid liability when it turns out your AI is biased against some protected class.
- foxyv 2y agoI think the place people get into the most trouble with hiring is when they use AI to filter applications. It's one thing to filter based on a specific criteria like education or experience, but to go by "AI didn't like the look of this resume" is something else.
- balls187 2y agoWhat AI systems are currently targeting my prostate? This space is primarily dominated by AI that replaces mundane work done by humans.
- nrfulton 2y agoThe AG's advisory is essentially a memo from the AG explaining how the AG thinks existing statute applies to a new technology. It's not new legislation or new case law, but it is worth paying attention to for various reasons. The section of the advisory referencing disproportionate impact is quoting, nearly word for word, a portion of Cal. Code Regs. Tit. 2, § 14027. So, this section of the advisory essentially amounts to the AG saying "using AI to do something illegal is still illegal". That does not really answer your question, though. > What kind of standard is disproportionate impact?... The kind that does has been codified in CA and other jurisdictions' statutes for a long while now. This means that the standard is extremely well-litigated in the state's courts, and so the answer to "what is disproportionate impact?" is, I think, something like: "That seems complicated; there's probably a rich case law that provides clarity in some situations but also highlights areas of ambiguity in other situations. If you're in it for profit, and have any questions, get a lawyer who specializes in that area of the law to review your specific circumstance; if you're in it for civics/curiosity, start with the statute then start reading significant case law or law reviews regarding that statute." It's also the kind of standard that can attract flame wars... hopefully not here, though ;-)
- alvah 2y agoMen aren’t a protected class anyway.
- kjkjadksj 2y agoFor the last point this made me think squarely at chatbots. I had a chatbox experience recently that could have been handled with a cancel order button on a website. I had to go through a phone tree with the chatbot, say “no” 3 times until it let me type a free response. Said I wanted to cancel an order. “Did you mean cancel order?” Yes. Yes I did. What should have been a 2 second interaction with a ui element became a 5 minute song and dance. And I am a reasonably competent person on a computer. For someone who isn’t maybe this is enough to make them turn away from whatever action they were attempting to do in frustration. That is a big impact. And depending on where these chatbots have been shoehorned in, that impact could easily be disproportionately burdened on a protected class.
- csa 2y ago> I had to go through a phone tree with the chatbot, say “no” 3 times until it let me type a free response. I’m not excusing this design (it could probably be improved), but it may have been intentional. See below. > And I am a reasonably competent person on a computer. Most people are not. It’s generally unwise to design an automated system that assumes computer/tech competence. > For someone who isn’t maybe this is enough to make them turn away from whatever action they were attempting to do in frustration. I imagine it’s the other way around. This type of system saves folks with less tech savvy from themselves. I’m not sure if you’ve designed systems like this before. I have, and I was very surprised at what people thought was reasonable interaction and/or reasonable input. Confirming choices, perhaps multiple times, before moving forward can save a lot of headache later for everyone involved. The collateral damage is guaranteed “wasted” time for everyone using the chatbot, but the company largely doesn’t care about that — that is, they are more than willing to pass on the cost of their money (e.g., if they hired a human agent) for your time.
- kjkjadksj 2y agoWe are talking about a cancel button here. If you can wrap your head around the order button they also have, you can wrap your head around a cancel order button. A phone tree is stupid lazy ux. Even worse when you now have to massage it out of a chatbot which isn’t even reliable in how it responds back to you like a phone tree used to be.
- arrosenberg 2y agoSo, once again, everything is securities fraud?
- Animats 2y ago- “Foster or advance deception” - definitely happening. Many (maybe most on some platforms) posts on social media are being generated by LLM-type AI systems. The question of whether a power tool for crime is aiding and abetting a crime is a tough one. The gun industry beat that. It has a lot to do with how the product is promoted. - "Falsely advertising the accuracy, quality, or utility of AI systems” - worst case is probably Tesla's Fake Self Driving. Most other LLM systems aren't allowed to make decisions, just blither. - "Create or sell an AI system or product that has an adverse or disproportionate impact on members of a protected class, or create, reinforce, or perpetuate discrimination or segregation of members of a protected class.“ - hm. Need more instances. Now, using computer systems to create a cartel to violate antitrust laws and push prices up is a thing. There's litigation against landlords for that. But that's not a "protected class" thing, it's an antitrust thing.
- lawlessone 2y agonot to seem pedantic but the self driving system isn't an LLM, it's probably using an artificial neural network for some aspects like object recognition (and would have similar issues like need large amounts of data) , but the similarity ends there.
- deleted 2y ago[deleted]