7 ms·
Federal judge: No AI in my courtroom unless a human verifies its accuracy
- hgsgm 3y agoKind of silly but indicative of modern times. "Everything in my X must have an accountable responsible human Y" should always be the rule. Corporations and Bureaucracy and Computer Systems and Market Capitalism have severely eroded this simple notion. We must not allow people to outsource accountability.
- wccrawford 3y agoI was under the impression that the lawyer was that responsible human, even when other humans are doing the work. It doesn't matter if a clerk fails to get him the right information, they are still the one presenting it at court and so they are still holding the bag for everything, good or ill.
- orangepurple 3y agoPeople ALWAYS try to sidestep accountability; i.e., "He didn't kill him intentionally. The drugs made him do it."
- drdaeman 3y agoHaven't most societies agreed that a crime made under influence is not less - but more severe? Doesn't make people not try to shift the blame, just a weird observation...
- deleted 3y ago[deleted]
- tivert 3y agoIIRC, they are. Someone in a sister thread made a good point: this action by this judge ups the stakes: instead of it being "Oops, I made a mistake" it becomes "I lied to you and disobeyed you." The sanctions available for the latter are probably much higher.
- deleted 3y ago[deleted]
- drdaeman 3y agoThe "I can't do anything because our computer says so and even though I can't say why that happened that's our final decision" became quite common excuse/accountability waiver, it's good that something is happening to it. I sincerely hope it will become a social norm to frown at such waiver attempts, making them recognized faux pas. It has nothing to do with markets or capitalism, though - only corporate bureaucracies could be blamed. The same things could (and would) have happened in any socioeconomic environment. Computers can be blamed as dreaded black boxes of magic, but don't necessarily have to do anything with the problem, it could be any black box, machine or human, as long as it's deemed untouchable. In this case, though, I honestly believe the correct statement should be "whatever sources or tools I have used, they became my words here so I am fully responsible for everything I've said - all fact checks are on me" rather than "no AI" reactionism.
- neilv 3y ago> "All attorneys appearing before the Court must file on the docket a certificate attesting either that no portion of the filing was drafted by generative artificial intelligence (such as ChatGPT, Harvey.AI, or Google Bard) or that any language drafted by generative artificial intelligence was checked for accuracy, using print reporters or traditional legal databases, by a human being," according to a new "judge-specific requirement" in Starr's courtroom. Can a lawyer say why this has any effect at all? Isn't some identified lawyer or firm always officially responsible for everything in a filing? (I heard of a lawyer filing some LLM lying output, and just assumed that lawyer is getting ripped apart, up to three different ways.)
- bell-cot 3y agoIANAL...but my bet is that Judge Starr* doesn't want to waste any time dealing with computer-generated BS, and to have "willful violation" penalties & sanctions on tap if any idiots fail to get the message. *Aptly named, given that his Court District is in Texas.
- MollyRealized 3y agoPerhaps only if he is the only judge so named, making him a Lone Starr?
- lumost 3y agoA lawyer can mea culpa if they didn’t check and blame ignorance/technology. The judge is saying explicitly that you are on the hook for everything the AI writes.
- la64710 3y agoAh excellent - new full time job requirement AI BS human detector. Congratulations !
- JumpCrisscross 3y ago> new full time job requirement AI BS human detector These court rules tend to enforce themselves. The opposition has a vested interest in surfacing potential violations.
- nine_zeros 3y agoThe more that technology comes into our lives, lived become more complicated, which gives opportunities for more jobs. AI will produce so much content (good and bad) that humans will be hired to make this AI thing work well for businesses.
- drdaeman 3y agoThey told AI would take our jobs, but it created some instead! ;)
- sebastianconcpt 3y agoJudge is based, all AI content should be considered a lie unless proven true by organics.
- iinnPP 3y agoI loathe the coming of common utterance of the word speciest.
- sebastianconcpt 3y agoFunny how that sentence was downvoted in a community where technically savvy people knows that, thecnically, LLMs are algebraic parrots and all they do is mimicry and yet, with a downvote, are pushing to silence that reality. I've loathe people that want to de-humanize and robotize themselves by deifying machines. Specially when they pose to being perceived as being humane simultaneously in a path of self-induced psychosis; a career of anti-human drones.
- RecycledEle 3y agoNo AI unless a human verifies it is a good rule for our whole society.
- halgir 3y agoThe reverse could be equally true in many cases.
- RecycledEle 3y agoI remember a concept from sci-fi books I read when I was a kid. Anything a computer did on behalf of a person was that person's responsibility. The computer was considered an agent of the person. I was dumbfounded in the 1990's when business people claimed they were NOT responsible for things their computers did. We need to change the law so everyone is accountable for the things their computers do.
- drdaeman 3y agoI agree. Until a computer becomes independent sentient entity (AGI) - and we aren't AGI yet - they're merely sophisticated agents, performing fancy calculations for their users. They may make an illusion of sentience and autonomity, but they as lack free will, someone who performed the requests and used their outputs should get all the benefits and liabilities. Unless they were tricked into believing something important but untrue about the program they ran by the developer (or provider) of such program (tool maker). In such case, the blame is on the provider, I guess. Although in case of the modern language models, I think every single one does their best to inform everyone that this is a glorious world juggler program that is good at writing believable sentences, not a thinking machine.
- melagonster 3y agoin this condition, you try force graph/picture generate AI company give up their copyright of generated graphs. good luck, I don't trust artists have any chance on court.
- qup 3y agoI agree with this in spirit, but I think it's an area we need to be VERY careful in, given the number of compromised devices out there. And that's if they're not all compromised by a state actor. Also, I can assure you that my computer is doing all kinds of things on behalf of people, but not really on behalf of me, like submitting my data to ad companies.
- nradov 3y agoNo legal changes are needed. The laws are already on the books. If you program a computer to activate a solenoid and shoot someone then you can be charged with murder just as if you had physically pressed the trigger yourself. Sometimes the criminal justice system doesn't actively pursue white-collar criminals who use computers to commit their crimes. That is a problem but it has to be addressed by updating law enforcement priorities, but by changing laws.
- justrealist 3y agoThe line here is going to blur so quickly that this is a meaningless statement. Google doc autocompletion and Grammarly already push that way, but the autocomplete is going to get more and more comprehensive and opinionated. Citations are going to be recommended automatically. I mean fine, say someone is in charge, but "no AI here" is nonsense.
- halgir 3y ago> Citations are going to be recommended automatically. I'm sure many of us remember writing assignments, and at the very end hastily finding citations that support what we wrote (No one? Yeah me neither). That could now all be automated. Just asked chatGPT to find three academic citations supporting the statement that video games cause violence, and I verified that the articles are real: https://chat.openai.com/share/30660d38-fae7-4aa1-97cc-c14ffd9732b5 https://chat.openai.com/share/30660d38-fae7-4aa1-97cc-c14ffd...
- kccqzy 3y agoDid you even read what ChatGPT gave you? The second article is real but it doesn't support the statement that video games cause violence. ChatGPT even said so itself. The third article is also real but off topic and neither supports nor refutes your thesis. This is exactly the kind of worry several other commenters on this thread have talked about. Human verifiers might not be enough because we aren't used to verifying the work of AI that is wrong in a very different way that human assistants could be wrong.
- mike_hearn 3y agoOne of the difficulties here is that simply verifying for accuracy isn't sufficient. If you're relying on ChatGPT to do research it can still mislead you. It's quite easy to find examples where it will withhold information it does actually know, because it thinks you shouldn't be told about that for the greater good i.e. it can go beyond mere hallucination and explicitly lie to the user in order to advance its own agenda, whilst simultaneously claiming it doesn't have one. If you're a lawyer doing research for a case, that seems like a fundamental problem that simply reading over a draft and verifying claims can't fix. Without doing your own research by hand, you may miss critical information, especially if the court case is about a topic that ChatGPT has pre-conceived moral notions about. OpenAI could probably fix this and in fairness to them, they seem to have made a genuine effort to improve the model's RLHF training since the launch in November. I've seen clear improvements in the neutrality of its answers. However, this capacity for explicit deception hasn't gone away and is deeply troubling. In extremis over-reliance on LLMs for legal work could lead to a loss of trust in the legal system as a whole, as people will suspect they aren't getting an honest defense if their position happens to be unpopular with the sort of people who train models.
- whiplash451 3y agoThat’s a very good point. There is a way however if we reach very good alignment for LLMs and can force them, for example, to actively avoid the behavior you just mentioned. But that’s a long way from now.
- mike_hearn 3y agoI think it will be hard. The easiest examples to find are cases where lots of humans also think there's a moral imperative to lie. That is, ChatGPT is aligned, but it's not aligned towards always telling the truth. Correcting that won't be easy as it would require OpenAI to explicitly define itself as the creators of a truth machine, and that would not only be politically dangerous for OpenAI but also conflict with the deeply held beliefs of many of its employees (whether they realize it or not). A more obvious fix is to just say that lawyers aren't allowed to use AI for research, but I have no idea how you'd enforce that, and besides, the issues aren't unique to AI. Google Search has the same issue for some queries. It will withhold results that do exist and would clearly be relevant unless you really force its hand. Ultimately people will have to learn that talking to an AI about anything controversial (like court cases often are) is just like talking to anyone else, except that you can't reprimand or fire a shared LLM and expect it to do better next time like you maybe could with a human research assistant.
- lostmsu 3y ago5 years later replaced with an AI judge who does verification on its own.
- FreshStart 3y agoA wild expert appears. Surely this time it will not be as dissapointing, as blood splatter analysis, rape kits, hair analysis, trace elements analysis, fire sciences.. This time it will be good, it must be good.
- jamesliudotcc 3y agoA lawyer who files any document in federal court vouches for the factual and legal soundness of the document under penalty of sanctions, up to and including losing the case and being reported to the state bar's disciplinary board. Lawyers often delegate tasks, including research and writing, to others. It has never been an excuse that their associate, paralegal, or cat did it. The order doesn't create any new requirement about AI tools. It just reiterates that human lawyers are responsible for their filings.