15 ms·
Lawyer cited 6 fake cases made up by ChatGPT; judge calls it “unprecedented”
- zugi 3y agoAm I the only one here entertained by the judge's clever wording? "Hey, all the case law you cited in your filing is made up. That's unprecedented!"
- CatWChainsaw 3y agoI don't see this as a particularly wild case of LLMs gone wrong. When "you have to take the bad with the good" includes a constant undermining of reality and an even more desperate need for fact-checkers, I wonder if it will have been worth it in the end. Likely not.
- mrangle 3y agoIndicates the inevitability of reduced time that it takes for fake history and data to be declared valid and widely defended by core institutions. Eventually the onslaught of subtle yet elaborate falsehoods will overwhelm the institutional filters.
- ftxbro 3y ago> "Lawyer: ChatGPT said the cases were real"
- sharemywin 3y agowe're really at this point aren't we.
- sidewndr46 3y agoI've always felt like Al Gore v. the Internet was a landmark case
- noman-land 3y agoAI Gore
- deleted 3y ago[deleted]
- glenngillen 3y agoOh boy, and this one! > The plaintiff's lawyer continued to insist that the cases were real. LoDuca filed an affidavit on April 25 in which he swore to the authenticity of the fake cases
- ftxbro 3y agomaybe they are trolling and their long game is to sue openai
- banana_giraffe 3y ago> In Shaboon v. Egyptair, 2013 IL App (1st) 111279-U. I'm sure a case involving Egyptair is complicated, still .. I'd love to see the 111,279 page volume this citation claims to come from.
- noman-land 3y agoThat's assuming it's citing the very last page of the document...
- ketralnis 3y agoDoubling down on your bullshit which is itself fuelled by a robot's bullshit is a move
- knifie_spoonie 3y ago> LoDuca filed an affidavit on April 25 in which he swore to the authenticity of the fake cases From the document another commenter linked above, it seems that affidavit is also dodgy: "The April 25 affidavit was filled in response to the Orders of April 11 and 12, 2023, but is sworn to before a Notary Public on the 25th of January 2023."
- boringg 3y agoDisbarred?
- Antoniocl 3y agoI would hope so, at a minimum! I'm shocked that signing an affidavit in a court of law certifying the existence of made-up courses isn't simply criminal.
- anamexis 3y agoIntent (mens rea) is a core part of what makes a crime a crime.
- anigbrowl 3y agoLook up 'strict liability'.
- kube-system 3y agoThat's an exception, not the general rule.
- anigbrowl 3y agoWhat of it? It exists in some cases, so the mens rea requirement is not universal.
- occamrazor 3y agoStrict liability is extremely rare in criminal matters.
- NoboruWataya 3y agoMen rea does not mean intent, it simply refers to the mental element of a crime. It can be intent or some lower bar such as recklessness or gross negligence, depending on how the crime is defined.
- 3y ago
- tromp 3y ago> The other five bogus cases were called Shaboon v. Egyptair, Petersen v. Iran Air, Martinez v. Delta Airlines, Estate of Durden v. KLM Royal Dutch Airlines, and Miller v. United Airlines. Tyler Durden no doubt...
- bitwize 3y agoSo turds turn up in AI law as well...
- ss108 3y agoIt would have taken about 1 minute to put each of them into a tool like Casetext, Lexis, WL, or Bloomberg Law to determine they didn't exist.
- onion2k 3y agoTo do that you first need to distrust AI, and a lot of people don't. They think of GPT like Google-but-written-in-English. That is a large part of the problem.
- ss108 3y agoWell, leaving aside whether a lot of people don't distrust and whether a lawyer would or should be among those people, lawyers typically don't cite things without checking them first. I work at a biglaw firm, can't imagine it never getting caught that the cases don't exist. This sounds like a midsize firm, should be fairly similar. There are a few small law firms/sole prop type guys, however, who I have crossed paths with for whom this kind of stupidity and carelessness would be on brand though. Guess he was just in a rush and figured this would be one of the 2/10 times he files something without at least taking a look at the opinions first, and it ended up being a massive error.
- anigbrowl 3y agoThat's not a valid excuse, though. Lawyers are paid big bucks to think, not to assume. Otherwise you could do your litigation for free by just asking interested people on Twitter. I went to law school and had to drop out due to an injury & attendant medical costs; it's a crime (as in going to jail) for me to practice law without being licensed, no matter how good my work product might be.
- bentcorner 3y agoThere are probably thousands of lawyers that thought about using ChatGPT for their profession, most of them realized it lies and never got farther than that, maybe a few hundred actually tried it out and also realized ChatGPT was lying, this is the one guy managed to swiss-cheese-model his way through.
- morkalork 3y agoImagine being that clown's client.
- thefourthchime 3y agoSigh, I'm getting very sick of hearing about how "ChatGPT" makes stuff up. Yes, 3.5 made a lot of stuff up, 4.0 still does, but it's much rarer. I wish people would mention this, it's all treated as the same thing. It's like talking about how unreliable these "Airplanes" are when they are talking about prop planes, even though jets are out.
- throwway120385 3y agoIf I had an airplane that usually flew just fine as long as I didn't use it for business, but would sometimes crash directly into the ground for no apparent reason, I would be talking about how unreliable it is too. Especially if the maker of the airplane touted its unusual benefits for searching for information in a business setting.
- tracerbulletx 3y agoNone of this reporting or line of criticism is good and I completely agree, but it is worthwhile to remember that most users are using the free version and so to most people that's what ChatGPT is.
- ok123456 3y agoMost people are using 3.5 since that's the free model.
- deleted 3y ago[deleted]
- dimgl 3y agoI'm so sick of seeing this strawman. "Well, ChatGPT 4 is better!" Well, how should I know? I'm not paying for ChatGPT 4. At least a product like Midjourney is consistent both in its paid and unpaid form.
- favorited 3y agoThese are generative conversation bots. Their #1 priority is to make things up.
- inopinatus 3y ago
- macmac 3y agoThat is a hilarious pun.
- rayiner 3y agoCheck out one of the fake opinions: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.29.1.pdf https://storage.courtlistener.com/recap/gov.uscourts.nysd.57.... It even makes up a panel comprising real federal appellate judges (although one is from the fifth circuit while the fake case is from the eleventh circuit). I can see how someone unfamiliar with what GPT can do could get fooled.
- greenyoda 3y agoThis is interesting. At the beginning of the document, we read that Susan Varghese, representing the estate of George Varghese, "appeals the district court's dismissal of her wrongful death claim". However, in the "Factual background" section, we read about someone named Anish Varghese who is alleging a breach of contract due to being denied boarding on a flight and having to purchase another ticket.
- cj 3y agoI asked ChatGPT to tell me a riddle. It was “What is always hungry, needs to be fed, and makes your hands red?” (Or something like that) I asked for a hint about 5 times and it kept giving more legitimate sounding hints. Finally I gave up and asked for the answer to the riddle, and it spit out a random fruit which made no sense as the answer to the riddle. I then repeated the riddle and asked ChatGPT what the answer was, and it gave me the answer (“Fire”) which makes sense as the answer to the riddle. But it was giving extremely bad hints, like “it starts with the letter P” and “it’s a fruit”. That was a great way to show my non-tech family members the limitations of AI and why they shouldn’t trust it. Playing “20 questions” with ChatGPT is another great way to expose its limitations. It knows the game and tries to play, but is terrible at asking questions to narrow down possible answers. There really needs to be some confidence or accuracy score/estimation displayed alongside its output. Or, learn how to say “I don’t know”
- anigbrowl 3y agoPlaying “20 questions” with ChatGPT is another great way to expose its limitations. It knows the game and tries to play, but is terrible at asking questions to narrow down possible answers. I began doing this last winter, and while it tends to be a bit slow I'm quite impressed that it can manage at all.
- Our_Benefactors 3y agoChat GPT versus Akinator!
- jimnotgym 3y ago> Or, learn how to say “I don’t know” This is the correct answer. It is like a sad salesman who is out of his depth, but decides to keep bullshiting!
- dylan604 3y agoor a politician. even when presented with incontrovertible evidence, double down
- rvba 3y ago> The lawyer's affidavit said he had "never utilized ChatGPT as a source for conducting legal research prior to this occurrence (...)" I wonder if the court tried to verify that.
- lynx23 3y agoWait, this is handled as "ChatGPT made up these cases" and not as "Layers deliberately used ChatGPT to fabricate stuff"? Is anyone really believing a lawyer is that stupid? I know, adssume good intentions and all, but in this case, really?
- dpiers 3y agoYou’re confusing stupidity with narrow expertise. I know CPAs who don’t file their own taxes because that wasn’t the focus of their niche. I know surgeons who don’t know the difference between a transistor and a resistor. Plenty of smart people don’t understand how ChatGPT works or what it’s limitations are. A bunch of nerds built the best BS generator in history and marketed it as a super intelligent computer. If you ask it for relevant cases and it spits out a bunch of plausible information, is it really on them to know the tool is just really good at making things up?
- droopyEyelids 3y agoA very intelligent person can simultaneously be very stupid, and stupid behavior can be increased with tight deadlines or a high workload.
- Projectiboga 3y agoIt was his first try with the tech. He should have popped the cases into West Law and at least skimmed the briefs, he is at a big firm with that subscription. I'm not a lawyer and I at least have a clue for the legal research shortcuts. This guy must have been drinking and or having an affair to have pit so little effort into his first try using a new tech. I'm guessing there is a risk of humorous legal parody getting scooped up by the chat-bot.
- lamontcg 3y agoMy bet would be that he cheated his way through life and the bar somehow and landed the job on the basis of his looks and presentation or who he knew or something like that, so asking ChatGPT and cheating on it was more natural than using actual legal research tools.
- mabbo 3y agoI think the judge should not take into consideration anything about where the lawyer said they got the case information from. When you go to court and cite previous cases, you are responsible for ensuring they are real cases. If you can't do that, what exactly is your job as a lawyer?
- kevinmchugh 3y agoIf someone hacked westlaw and maliciously replaced opinions, I think it would not warrant censure of lawyers who cited westlaw. Maybe a bad day with the judge, and they'd have to amend the filings. This is malpractice in a way that relying on an accepted if technically fallible resource isn't
- mabbo 3y agoWhat a great plot that would be for a movie. Big court case where it's pretty clear the plucky good guys aren't going to win it. So they hire hackers to break into westlaw and give the opposition fake cases, then bait them into presenting those cases - and call them out in it.
- dylan604 3y ago>If you can't do that, what exactly is your job as a lawyer? to convince some (judge/jury) that your client is the one to win the trial. ethics are meant to be the definer of how far to go in that cause with licensing boards being the ultimate decider if you've crossed the line and are allowed to continue in the legal practice.
- deleted 3y ago[deleted]
- inopinatus 3y agoGrinding against the safety rails on a mountain hairpin is not indicative of a competent driver.
- 3y ago
- latexr 3y agoPrior discussion on the same matter from a different link: https://news.ycombinator.com/item?id=36095352 https://news.ycombinator.com/item?id=36095352
- dang 3y agoThanks! Macroexpanded: Lawyer cites fake cases invented by ChatGPT, judge is not amused - https://news.ycombinator.com/item?id=36097900 https://news.ycombinator.com/item?id=36097900 - May 2023 (304 comments) A man sued Avianca Airline – his lawyer used ChatGPT - https://news.ycombinator.com/item?id=36095352 https://news.ycombinator.com/item?id=36095352 - May 2023 (127 comments) ChatGPT-Authored Legal Filing “Replete with Citations to Non-Existent Cases" - https://news.ycombinator.com/item?id=36092509 https://news.ycombinator.com/item?id=36092509 - May 2023 (71 comments) Presumably also related from earlier today: Mandatory Certification Regarding Generative Artificial Intelligence - https://news.ycombinator.com/item?id=36131942 https://news.ycombinator.com/item?id=36131942 - May 2023 (35 comments)
- vanattab 3y agoSounds like the man's real case is against his lawyer.
- hluska 3y agoFor reference, here is the judge’s order. Endnote #2 alone is worth the price of admission: https://s3.documentcloud.org/documents/23826753/judgeaskingtheotherlawyerwhyhesubmittedafilingwithfakecases.pdf https://s3.documentcloud.org/documents/23826753/judgeaskingt...
- StrangeATractor 3y agoThat guy has some work to do on his resume.
- josu 3y agoHe'll just outsource it to chatGPT.
- not2b 3y agoAt minimum the sanctions should include a fine sufficient to cover the costs of everyone who had to deal with this: the court, opposing attorneys. Maybe punitive damages too. But let the guy keep his law license (unless there's a second offense).
- CSMastermind 3y agoIt feels like there definitely should be punitive damages? It's like they didn't even check the text that ChatGPT generated for correctness.
- seydor 3y agothis is funny. However GPTs are great linear interpolators between stories and probably also between judgements. Maybe they will be useful to replace judges
- kristianbrigman 3y agoHow do these usually get checked? I mean, maybe he’s already done this successfully a few times.
- ajb 3y agoThey get checked by the lawyer for the other side.
- pkilgore 3y agoUsually the first thing a Judge's clerk will do before a judge reads a brief is pull all of the authority cited for the judge to have ready-to-review along side the brief itself (if not analyzed, highlighted, summarized, etc. etc. depending on the judge). Now, whether the judge actually reads them is debatable (I had my doubts sometimes). But you bet your ass that if the Clerk simply cannot find a case, the Judge will be informed of that. YMMV in State courts, which can be all over the place in terms of professionalism. But you should at least assume your opponent is going to read your cases because the easiest way to beat someone in court is to point out the law you rely on is bad.
- deleted 3y ago[deleted]
- onionisafruit 3y ago"unprecedented"? ChatGPT says there is precedent and gave me several citations.
- CatWChainsaw 3y agoThis is a joke right? The citations were made up too.
- xbar 3y agoI asked ChatGPT to disbar an attorney.
- 0xbadcafebee 3y agoWere they any worse than the normal junk cases that get filed every day in this country?
- Brendinooo 3y agoYeah, AI-generated fake cases would definitely be unprecedented, unless SmarterChild did a stint as a paralegal in the early aughts...
- Ankaios 3y agoIt turns out there were precedents: Case: Thompson v. Horizon Insurance Company, Filing: Plaintiff's Motion for Class Certification. Citation: The plaintiff's attorney cites the influential case of Johnson v. Horizon Insurance Company, 836 F.2d 123 (9th Cir. 1994), which established the standards for certifying class actions in insurance disputes. However, it has recently come to light that Johnson v. Horizon Insurance Company is a fabricated case that does not exist in legal records. Case: Rodriguez v. Metro City Hospital, Filing: Defendant's Motion to Exclude Expert Testimony. Citation: The defense counsel references the landmark case of Sanchez v. Metro City Hospital, 521 U.S. 987 (2001), which set the criteria for admitting expert witness testimony in medical malpractice cases. However, it has now been discovered that Sanchez v. Metro City Hospital is a fictitious case and does not form part of legal precedent. Case: Barnes v. National Pharmaceuticals Inc., Filing: Plaintiff's Response to Defendant's Motion for Summary Judgment. Citation: The plaintiff's lawyer cites the well-known case of Anderson v. National Pharmaceuticals Inc., 550 F.3d 789 (2d Cir. 2010), which recognized the duty of pharmaceutical companies to provide adequate warnings for potential side effects. However, further investigation has revealed that Anderson v. National Pharmaceuticals Inc. is a fabricated case and does not exist in legal jurisprudence.
- freejazz 3y agoIn any of those examples, when pointed out by the court that the cases do not seem to exist, did the attorney's then go and print out a fake case and file it with the court? That's what the atty here did. It's a bit further than what you list.
- mensetmanusman 3y ago"He was a bad lawyer. I am a good Bing. I will send him to jail."
- Zetice 3y agoSo how do LLMs fit into the legal profession, if at all? Do legal tools that make use of LLMs just need to come with big ol' disclaimers at the top saying, "This tool does not represent a legal opinion, please verify the output independently."?
- alfalfasprout 3y agoAt minimum, attorneys need to review the work's citations and wording for accuracy. At the end of the day, this is not too different than LLMs consistently writing subtly broken code-- someone needs to comb through it and fix it. We're currently in the phase where the potential of LLMs is suddenly appealing to many but where most people don't quite understand it's not really magic and that even after they evolve they will remain critically flawed. Expect serious growing pains as a result.
- clnq 3y agoThe same way they fit into all professions demanding accountability and precision - they can be used for exploration but everything they say should be verified.
- delecti 3y agoI think LLMs should be used with basically the same stipulations in any field. The words it outputs are usually valid English, but not necessarily accurate, so it's good for brainstorming but needs to be fact-checked. Overall, whether it's useful depends on whether the time required for the latter is less than the time you save with the former. Personally, I've found them most useful as a way to provoke myself into Cunningham's Law, essentially relying on the fact that they make shit up. https://meta.wikimedia.org/wiki/Cunningham%27s_Law https://meta.wikimedia.org/wiki/Cunningham%27s_Law
- uguuo_o 3y agoI thought I had just seen a post yesterday [0], so I immediately thought how can there be 2 such lawyers? Competition on who would get caught first? It seems to be referring to the same case. [0] https://news.ycombinator.com/item?id=36116652 https://news.ycombinator.com/item?id=36116652
- more_corn 3y agoEvery citation created by chat gpt will be hallucinated. It knows what they look like. It doesn’t know what they are. It doesn’t actually “know” anything. It is a statistical engine for generating the next reasonable looking word.
- rgoulter 3y agoI asked an LLM "during the day, what colour is the sky?", and it told me the sky is typically blue, and that this has to do with sunlight scattering as it enters our atmosphere. I think it's worth using "hallucinate" to more precisely refer to inaccurate content. I'd think of this in the same sense as a false-positive from some test result. I like the term "bullshit" when used to mean "made without regard to the truth".
- thih9 3y ago> Schwartz didn't previously consider the possibility that an artificial intelligence tool like ChatGPT could provide false information Don’t you have to click through a number of popups about that before accessing chatgpt?
- CatWChainsaw 3y agoDo you painstakingly read the EULA/ToS of every service you use that has one? Because you're in a slim minority if you do.
- thih9 3y agoIt’s not eula/tos though. > This is a free research preview. > Our goal is to get external feedback in order to improve our systems and make them safer. > While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice. This is the complete text of the first popup. There are three, each with a bit of text highlighting that this is an experimental service. There are emoji and there is an alert emoji next to the “incorrect/misleading” bit. They show up every time you visit.