11 ms·

Two US lawyers fined for submitting fake court citations from ChatGPT

213 points by gorbachev 3y ago

gorbachev 3y ago
re: https://news.ycombinator.com/item?id=36242462 https://news.ycombinator.com/item?id=36242462
damnesian 3y ago
Everyone letting ChatGPT do their jobs, and the people training ChatGPT are too lazy to do the work themselves, they actually use ChatGPT to train ChatGPT... what could possibly go wrong in this scenario?
- brucethemoose2 3y ago
  I wouldn't say OpenAI is "lazy," but this is still an excellent point about LLMs. The researchers training the ones we use now are in a massive hurry. They aren't meticulously curating datasets, checking usage rights, hiring a huge field of human curators, or thoroughly field testing them. They are madly working on a new improvement, publishing it in a paper, then moving onto the next thing. This has been a cultural issue in AI land for years. These experimental models were never meant for the general public, yet here we are.
- qayxc 3y ago
  Related: https://arxiv.org/abs/2305.17493v2 https://arxiv.org/abs/2305.17493v2 Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet. It could be all downhill from hereon out, as the amount of AI generated content will soon overtake actual human generated and -curated data on the internet. So the party could be over soon ¯\_(ツ)_/¯
sgbeal 3y ago
That should be a career-ending move, not a fine.
- mindcrime 3y ago
  I disagree with the notion that it should be "career ending", but I do think they got off pretty light.
  - Ensorceled 3y ago
    They doubled down on their original mistake rather than admit what they did, even when it was quite clear the judge and opposing council was on to them. They should be disbarred.
    - s1artibartfast 3y ago
      Where is this doubling down? It certainly isn't in this article.
      - isp 3y ago
        See https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_5.pdf https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... - Page 2, the judge uses the very words "doubled down": "But if the matter had ended with Respondents coming clean about their actions shortly after they received the defendant’s March 15 brief questioning the existence of the cases, or after they reviewed the Court’s Orders of April 11 and 12 requiring production of the cases, the record now would look quite different. Instead, the individual Respondents doubled down and did not begin to dribble out the truth until May 25, after the Court issued an Order to Show Cause why one of the individual Respondents ought not be sanctioned."
        s1artibartfast 3y ago
        I guess that really depends on what you consider doubling down. -On March 1, They filed a submission with fake citations from chatGPT. -On April 11th court told them the cases couldnt be found and to produce full transcripts of the cases by April 25. -On April 25, They came back and said they don't have them and couldnt find the cases, only the limited excerpts provided in "online databases". -On May 4, the court asked them why they ought not be sanctioned for citing non-existent cases to the Court. -On May 25, They said we found these cases on ChatGPT and thought they were real. I guess the more transparent response on April 25 would have been to say "We weren't able to find the full cases and we now believe they probably dont exist ". Im not sure if the latter part is material if they already admitted they cant find the cases. If you cant produce the cases, is the reason relevant at that point?
curiousgal 3y ago
Your honour I am not a cat
- Galacta7 3y ago
  I can see that... :P
- justincredible 3y ago
  [dead]
- dennis_jeeves1 3y ago
  What evidence do you have?
danielmarkbruce 3y ago
Speaking confidently in a structured manner and sprinkling in 'facts' has always been quite convincing to humans.
isp 3y ago
Related: - "A man sued Avianca Airline – his lawyer used ChatGPT" (27 days ago, 139 comments): https://news.ycombinator.com/item?id=36095352 https://news.ycombinator.com/item?id=36095352 - "Lawyer who used ChatGPT faces penalty for made up citations" (15 days ago, 128 comments): https://news.ycombinator.com/item?id=36242462 https://news.ycombinator.com/item?id=36242462 - "Lawyers blame ChatGPT for tricking them into citing bogus case law" (14 days ago, 175 comments): https://news.ycombinator.com/item?id=36257545 https://news.ycombinator.com/item?id=36257545 Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca-inc/ https://www.courtlistener.com/docket/63107798/mata-v-avianca... The full document from the federal judge on sanctions: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_5.pdf https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... One of many highlights: dismissing the lawyer's bad-faith argument with a footnote quoting Alice in Wonderland (footnote 12, page 18).
- cj 3y ago
  Are these all reporting on the same few lawyers or are these stories all isolated/unique cases?
  - isp 3y ago
    All the same case.
BlakeSimpson 3y ago
It blows my mind that a lawyer would go through all the schooling and DEBT just to use ChatGPT and risk losing their BAR card. Sure, I see plenty of uses for ChatGPT in their field, but to not even check the output before submitting it is what blows my mind.
- hospitalJail 3y ago
  They didn't know it could lie? But also, pretty sure Doctors and Lawyers are among the least trusted/highest corruption professions. All that schooling is a barrier to entry to keep outsiders away, it doesnt guarantee competence. Basically once you get in, graduation rates are close to 100%.
  - sarchertech 3y ago
    You’re very wrong about the perception of doctors. According to polls, doctors are the 2nd most trusted profession. https://news.gallup.com/poll/388649/military-brass-judges-among-professions-new-image-lows.aspx https://news.gallup.com/poll/388649/military-brass-judges-am... My wife runs the resident eduction for her division and they are very aware of the dangers of sending an incompetent doctor out into the world to practice. They’ll force them through extra training and make them repeat residency if they have to. Residents also drop out of their programs and move into other ones. It’s rare for them to drop out completely because they’d still be on the hook for $200k+ in loans. But medicine is a big field and it’s rare that someone who made it into and through med school to not be able to find something they are competent and capable at. Maybe you’re not great with patients, but you can look through a microscope all day as a pathologist etc… Incompetent and corrupt doctors still make it through the process of course, but far far fewer than in our profession and just about any other profession I can think of. As for lawyers: Law schools have around a 77% graduation rate nation wide. And about 90% of law school graduates eventually pass the bar exam. So we’re look at somewhere around 70% of people making it through after a pretty selective filter to begin with.
    - echelon 3y ago
      > According to polls, doctors are the 2nd most trusted profession. n=1 anecdote here: When I was in research track biochem and bumped into premed students, nearly the entire population of premed didn't seem to care about the science behind what they were learning. They formed a very different social clique. The "top students" in that set were sharing previous years' organic chemistry exams and none of them would read papers or get involved in research. They were entirely disinterested in the science. In my interactions with general practitioners, I like to talk about medicines' method of actions, pharmacology, the actual biochemistry behind diseases. Most of them seem to have no clue or retention of this information. I'm not trying to challenge them, either -- I'm generally curious to learn. That's not to say all doctors are like that. A lot of the surgeons and specialists I know are walking tomes of information.
hospitalJail 3y ago
I had someone run into basically the same issue with their dissertation. I went back into my texts since I had recommended them chatGPT. I screenshotted the part where I said: "You can't trust it for valid information" I even told him that everyone needs to be punished by incorrect information from ChatGPT once to really understand what its capable of. You need that traumatic event. Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually?
- s1artibartfast 3y ago
  >Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually? Because other people's thoughts are usually wrong so you cant trust them.
  - romwell 3y ago
    Sounds exactly like something chatGPT would say to convince you to trust it :D
- add-sub-mul-div 3y ago
  Why can't people understand human nature? Why can't people just do research the real way rather than circuitously go to the wrong answer machine and then double check everything it says?
- NoMoreNicksLeft 3y ago
  > Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually? If you've evolved from a species that is profoundly stupid, as a rule, then perhaps an adaptation/mutation that makes you mistrusting of that species advice is one that tends to propagate. For every "don't use ChatGPT, it's unsafe", there's a "I talk to magical beings who tell me you're not allowed to eat good food and you should mutilate your genitals". Statistically, refusing to learn from other people is probably the best bet.
notavalleyman 3y ago
> Levidow, Levidow & Oberman said in a statement on Thursday that its lawyers “respectfully” disagreed with the court that they had acted in bad faith. “We made a good-faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth,” it said. Seems like they still don't get it. It's not good faith representation of your client, it's not good lawyering to interrogate a chatbot for your case. Even if it could somehow only ever tell the truth. It's not a "good faith mistake" - so I think they still don't understand the problem here. It might be representative of greater society: when people interact with ChatGPT, they treat it as an oracle. I see it all the time on r/chatgpt - people say, "I asked ChatGPT about my long lost grandparent whose life I am researching. Apparently my grandma was a pilot!" Seems like the lawyers are shocked the oracle lied but still think they were right to consult it.
- enumjorge 3y ago
  The cynic in me thinks they do get it and that their statement is an excuse to save face. It’s easier to blame technology instead of admit they wanted to cut corners.
- bitshiftfaced 3y ago
  > Seems like they still don't get it. Oh they get it. ChatGPT explained it to them, and then suggested that they take this position because it was the least bad option.
  - brewdad 3y ago
    Right. They aren't completely finished with potential punishments. A review by the Bar is likely still ongoing. They can't do anything but pretend it was all done in good faith. Much like a person or company who may end up in a lawsuit can't issue a "real" apology when they screw up lest it be used against them.
    - mirekrusin 3y ago
      I bet a beer they came up with this defense line with a help of gpt.
HumblyTossed 3y ago
Do they just not understand how ChatGPT works? Do they think it's a search engine?
- bell-cot 3y ago
  "No" and "yes", evidently.
- asylteltine 3y ago
  [dead]
- AHOHA 3y ago
  You will be surprised how someone who’s not techie, doesn’t understand tech, heck, I even worked with people who supposedly are tech savvy but lack the basic of the basics. But overall, lawyers and doctors are the least knowledgeable in tech from my experience, I found farmers for example relatively are better, maybe because of a mindset that he/she needs to learn tech to navigate around by themselves, while a lawyer is snob a out it or just hire someone else to do the whole work.
- stefan_ 3y ago
  Every other HN thread on ChatGPT: artificial general intelligence is near! Isn't this just the same thing as the brain? GPT4 is a game changer! This thread: do they not know what ChatGPT is? Did they not verify the output? How could they believe this? Just maybe when an army of grifters jumps on the next hottest AI thing to advance their careers that kind of bald-faced lying has consequences?
  - olddustytrail 3y ago
    Can you genuinely not tell the difference between "this new technology is amazing" and "you should take everything this model says as gospel"? Because if you can't, it's not the fault of "grifters"...
AnIdiotOnTheNet 3y ago
I think many commenters here are unaware of exactly how forehead-slappingly WTF it is that these lawyers did. Even if ChatGPT were replaced with a well respected human legal expert saying exactly the same things, they still cited case law that they couldn't verify the existence of, let alone that the context of the ruling was actually helpful to their cause. Then when a judge told them "hey, we can't find any of these cases, what the hell is going on?" they doubled down. They're basically saying "trust me bro, these rulings totally happened and they totally say we're right".
- jeroenhd 3y ago
  When criticized about the made up cases they even showed (undated) screenshots of them asking ChatGPT if the cases were real. This is like your local major getting tricked into buying the Eiffel tower using public funds and then defending himself by saying he asked the guy he bought the Eiffel tower from and the guy said he was totally legit.
jeroenhd 3y ago
LegalEagle did a video on this case two weeks ago, digging into it in some more depth: https://youtu.be/oqSYljRYDEM https://youtu.be/oqSYljRYDEM They lied to a judge, doubled down, showed further incompetence by admitting to not knowing how the legal reference they supposedly quoted was even structured, then lied to a judge again trying to get more time to get their story straight, and all it cost them was $5000? And even then they have the guts to say they disagree with the judge? Levidow, Levidow & Oberman must either be run by fools or take the rest of the world for fools. I can understand how the rubber stamping lawyer signing all the paperwork got off easy; he was just signing documents because the lawyer who supposedly did all the work for the case couldn't operate in the court the case was transfered to. The other lawyer, the one who did the AI equivalent from copy pasting from the summary page of Google, should not be deemed fit to practice law.
- loeg 3y ago
  Also good coverage of the case on Serious Trouble (podcast): https://podcasts.apple.com/us/podcast/the-legal-brief-beyond-the-machine/id1630160928?i=1000615895028 https://podcasts.apple.com/us/podcast/the-legal-brief-beyond...
- jcranmer 3y ago
  > and all it cost them was $5000? They also have to write written apologies to their client, and all of the judges who were mentioned in their fake cases, with proof filed in the docket. And they are the brunt of a very well-publicized lawsuit. The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar already knows about it. (The judge has no power to disbar them himself, he can only refer to the bar for disbarment.) The biggest surprise is that they didn't get made to pay the defendant's legal fees as well, but that's because defendant didn't ask for any fees. Oh, and the final punishment... they lost the case--they get no chance to redo their opposition to the motion to dismiss. Arguably, their client can also sue them for malpractice, and their malpractice carrier is likely to prefer to settle that rather than actually fight the case.
  - 3y ago
jawns 3y ago
Every version of ChatGPT that I have used, going back to the beginning of January 2023, has had a disclaimer appear when you first load up the page that specifically tells you, "May occasionally generate incorrect information." This is why the law firm's contention that they were acting in good faith is laughable. When the tool itself shows a prominent disclaimer that it "May occasionally generate incorrect information," then the law firm can't reasonably argue that "We made a good-faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth." It was either gross negligence or intentional malfeasance. The most interesting thing to me is why the firm is continuing to try to excuse its actions, rather than taking full responsibility for them. I guess they assume that the classic Shaggy "It Wasn't Me" defense is less likely to lose them business, because they think so little of the clients they're trying to attract that they're wagering they'll buy it.
stblack 3y ago
That's a shockingly low fine.
blindriver 3y ago
There are A LOT of VERY bad lawyers out there. I have a friend who used to be a federal judge for immigration, and he said that immigration lawyers are especially the worst, because families need their services desperately, and these lawyers will take their money and spend NO time on their cases. They show up to court and are figuring stuff out real time. He tore one lawyer a new asshole for doing exactly this for his client and ordered them to come back better prepared. So the idea that some of them will use ChatGPT to do their work doesn't surprise me at all.
- ianhawes 3y ago
  I work in the immigration space. Part of the problem is that because immigration is technically federal but operates almost entirely in administrative venues (i.e. USCIS and EOIR), immigration lawyers only need to be barred in their home state and do not generally need admission to a relevant federal court bar or appear pro hac vice to represent clients in immigration court or even to make an appearance in filings.
  - hgsgm 3y ago
    How is that relevant? Incompetent / criminal lawyers are almost never disbarred in any field.
- pclmulqdq 3y ago
  It's also not usually a lot of money in comparison to their alternatives. I once had a criminal defense lawyer tell me that they make up the difference with scale and essentially batch-process people.
User23 3y ago
There’s probably a giant opportunity for pro se litigants here. Several possibilities suggest themselves.
jlarocco 3y ago
I wonder if this case was a setup for a lawsuit against OpenAI? Given the marketing and the way Microsoft and Google were tripping over themselves to "catch up", the impression in the media and to the general public is that chat bots are a magic silver bullet for everything. Even a lot of tech people are unclear about exactly how these things work. The case seems pretty clear: "They said it was artificial intelligence, but it wasn't very intelligent to make up cases. Look at the damages it's caused me..."
- sh34r 3y ago
  “You even said ChatGPT passed the bar exam!” As entertaining as this is to ponder, I don’t think they’re that clever. I don’t think they’d do so much damage to their reputation for such a flimsy case. They’re just incompetent.
lionkor 3y ago
I dont understand - ChatGPT says that it outputs inaccurate information RIGHT BELOW THE PROMPT BOX?! Illiterate lawyers?!
dctoedt 3y ago
In court, good lawyers think of themselves as "officers of the court" — to use a sometimes-forgotten phrase — who are working not just for their clients but also for the judge, to help the judge (and the judge's law clerks) to get it right, with rulings that will stand up on appeal if it comes to that. The good lawyer's job is not just to advocate the client's position — it's also to do "completed staff work" for the court in the process, even acknowledging any weaknesses in the client's case (which builds credibility). As we've seen in the ChatGPT case, judges tend to respond badly when they think a lawyer is trying to put one over on them. And conversely, judges often respond favorably when they perceive that a lawyer is being thorough and intellectually honest, not merely throwing up any bullshit argument that comes to mind. A fond memory from years ago is when a federal judge called me and my opponent — a well-known lawyer in our field — into his chambers to talk about a pending motion that involved an uncertain point of law, which we had extensively briefed. My opponent had not done an especially-great job; my own briefing had acknowledged the weak points in my client's position but also explained why my client should win anyway. The judge said, I don't really know what the answer is here. He then turned to me and said, But I'm more comfortable with your analysis. And he ruled my way – which won the case for my client (a company you might well be a customer of today).
thrw_aus 3y ago
So I guess ChatGPT didn't actually pass the bar exam. I wonder when tech firms will actually be held responsible for false advertisement.
OliverJones 3y ago
And every time this happens from now on, the fine should be doubled. When the fine gets too high for the lazy lawyer to pay, they may instead relinquish their bar admission (lose their license to practice law) for five years. Lawyers admitted to the bar aren't innocents. They're "officers of the court", meaning the legal system relies on them to avoid BS in their citation of other cases and of statutes and laws. It's never been acceptable for a lawyer to sign off on something without reading it and making sure it's right. What if registered professional civil engineers started asking generative AI for sewer-system or bridge designs and then didn't inspect them before approving them and handing them off to the guys with dozers and cement trucks? Not only would people fall into rivers, but the rivers would be polluted. The real world consequences are similarly serious when lawyers make stuff up, or let their interns or word-processing software make stuff up.
karim79 3y ago
It would seem that in the future, people will be less inclined to proof-read their work, or even read it at all prior to submission. I'd wager a guess that this will only get worse over time. LLMs will do more stuff for people - from helping school children to get through their homework, to higher learning assignments and all the way up to the "real" jobs. Along the way people might just place far too much trust in these tools. I'm pretty sure that's something we can all agree on. What I do find striking here though, is the incredible laziness of the lawyers. Not even checking whether or not the cases actually happened? That's just bad lawyering. The AI actually helped the system to call them out on their bad lawyering. That's not "bad things happening because AI" but just plain stupidity.
v3ss0n 3y ago
They just need to become a better prompt engineer. Send them to Andrew Ng course
leoradovey1993 3y ago
[dead]

213 points by gorbachev 3y ago

re: https://news.ycombinator.com/item?id=36242462 https://news.ycombinator.com/item?id=36242462

Everyone letting ChatGPT do their jobs, and the people training ChatGPT are too lazy to do the work themselves, they actually use ChatGPT to train ChatGPT... what could possibly go wrong in this scenario?

brucethemoose2 3y ago
I wouldn't say OpenAI is "lazy," but this is still an excellent point about LLMs. The researchers training the ones we use now are in a massive hurry. They aren't meticulously curating datasets, checking usage rights, hiring a huge field of human curators, or thoroughly field testing them. They are madly working on a new improvement, publishing it in a paper, then moving onto the next thing. This has been a cultural issue in AI land for years. These experimental models were never meant for the general public, yet here we are.
qayxc 3y ago
Related: https://arxiv.org/abs/2305.17493v2 https://arxiv.org/abs/2305.17493v2 Basically, if this paper is right, 2021 was the last year usable training data could be pulled from the internet. It could be all downhill from hereon out, as the amount of AI generated content will soon overtake actual human generated and -curated data on the internet. So the party could be over soon ¯\_(ツ)_/¯

sgbeal 3y ago

That should be a career-ending move, not a fine.

mindcrime 3y ago
I disagree with the notion that it should be "career ending", but I do think they got off pretty light.
- Ensorceled 3y ago
  They doubled down on their original mistake rather than admit what they did, even when it was quite clear the judge and opposing council was on to them. They should be disbarred.
  - s1artibartfast 3y ago
    Where is this doubling down? It certainly isn't in this article.
    - isp 3y ago
      See https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_5.pdf https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... - Page 2, the judge uses the very words "doubled down": "But if the matter had ended with Respondents coming clean about their actions shortly after they received the defendant’s March 15 brief questioning the existence of the cases, or after they reviewed the Court’s Orders of April 11 and 12 requiring production of the cases, the record now would look quite different. Instead, the individual Respondents doubled down and did not begin to dribble out the truth until May 25, after the Court issued an Order to Show Cause why one of the individual Respondents ought not be sanctioned."
      - s1artibartfast 3y ago
        I guess that really depends on what you consider doubling down. -On March 1, They filed a submission with fake citations from chatGPT. -On April 11th court told them the cases couldnt be found and to produce full transcripts of the cases by April 25. -On April 25, They came back and said they don't have them and couldnt find the cases, only the limited excerpts provided in "online databases". -On May 4, the court asked them why they ought not be sanctioned for citing non-existent cases to the Court. -On May 25, They said we found these cases on ChatGPT and thought they were real. I guess the more transparent response on April 25 would have been to say "We weren't able to find the full cases and we now believe they probably dont exist ". Im not sure if the latter part is material if they already admitted they cant find the cases. If you cant produce the cases, is the reason relevant at that point?

curiousgal 3y ago

Your honour I am not a cat

Galacta7 3y ago
I can see that... :P
justincredible 3y ago
[dead]
dennis_jeeves1 3y ago
What evidence do you have?

danielmarkbruce 3y ago

Speaking confidently in a structured manner and sprinkling in 'facts' has always been quite convincing to humans.

isp 3y ago

Related: - "A man sued Avianca Airline – his lawyer used ChatGPT" (27 days ago, 139 comments): https://news.ycombinator.com/item?id=36095352 https://news.ycombinator.com/item?id=36095352 - "Lawyer who used ChatGPT faces penalty for made up citations" (15 days ago, 128 comments): https://news.ycombinator.com/item?id=36242462 https://news.ycombinator.com/item?id=36242462 - "Lawyers blame ChatGPT for tricking them into citing bogus case law" (14 days ago, 175 comments): https://news.ycombinator.com/item?id=36257545 https://news.ycombinator.com/item?id=36257545 Original court documents: https://www.courtlistener.com/docket/63107798/mata-v-avianca-inc/ https://www.courtlistener.com/docket/63107798/mata-v-avianca... The full document from the federal judge on sanctions: https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0_5.pdf https://storage.courtlistener.com/recap/gov.uscourts.nysd.57... One of many highlights: dismissing the lawyer's bad-faith argument with a footnote quoting Alice in Wonderland (footnote 12, page 18).

cj 3y ago
Are these all reporting on the same few lawyers or are these stories all isolated/unique cases?
- isp 3y ago
  All the same case.

BlakeSimpson 3y ago

It blows my mind that a lawyer would go through all the schooling and DEBT just to use ChatGPT and risk losing their BAR card. Sure, I see plenty of uses for ChatGPT in their field, but to not even check the output before submitting it is what blows my mind.

hospitalJail 3y ago
They didn't know it could lie? But also, pretty sure Doctors and Lawyers are among the least trusted/highest corruption professions. All that schooling is a barrier to entry to keep outsiders away, it doesnt guarantee competence. Basically once you get in, graduation rates are close to 100%.
- sarchertech 3y ago
  You’re very wrong about the perception of doctors. According to polls, doctors are the 2nd most trusted profession. https://news.gallup.com/poll/388649/military-brass-judges-among-professions-new-image-lows.aspx https://news.gallup.com/poll/388649/military-brass-judges-am... My wife runs the resident eduction for her division and they are very aware of the dangers of sending an incompetent doctor out into the world to practice. They’ll force them through extra training and make them repeat residency if they have to. Residents also drop out of their programs and move into other ones. It’s rare for them to drop out completely because they’d still be on the hook for $200k+ in loans. But medicine is a big field and it’s rare that someone who made it into and through med school to not be able to find something they are competent and capable at. Maybe you’re not great with patients, but you can look through a microscope all day as a pathologist etc… Incompetent and corrupt doctors still make it through the process of course, but far far fewer than in our profession and just about any other profession I can think of. As for lawyers: Law schools have around a 77% graduation rate nation wide. And about 90% of law school graduates eventually pass the bar exam. So we’re look at somewhere around 70% of people making it through after a pretty selective filter to begin with.
  - echelon 3y ago
    > According to polls, doctors are the 2nd most trusted profession. n=1 anecdote here: When I was in research track biochem and bumped into premed students, nearly the entire population of premed didn't seem to care about the science behind what they were learning. They formed a very different social clique. The "top students" in that set were sharing previous years' organic chemistry exams and none of them would read papers or get involved in research. They were entirely disinterested in the science. In my interactions with general practitioners, I like to talk about medicines' method of actions, pharmacology, the actual biochemistry behind diseases. Most of them seem to have no clue or retention of this information. I'm not trying to challenge them, either -- I'm generally curious to learn. That's not to say all doctors are like that. A lot of the surgeons and specialists I know are walking tomes of information.

hospitalJail 3y ago

I had someone run into basically the same issue with their dissertation. I went back into my texts since I had recommended them chatGPT. I screenshotted the part where I said: "You can't trust it for valid information" I even told him that everyone needs to be punished by incorrect information from ChatGPT once to really understand what its capable of. You need that traumatic event. Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually?

s1artibartfast 3y ago
>Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually? Because other people's thoughts are usually wrong so you cant trust them.
- romwell 3y ago
  Sounds exactly like something chatGPT would say to convince you to trust it :D
add-sub-mul-div 3y ago
Why can't people understand human nature? Why can't people just do research the real way rather than circuitously go to the wrong answer machine and then double check everything it says?
NoMoreNicksLeft 3y ago
> Gosh, why can't people just listen and accept other people's thoughts. Why must we learn individually? If you've evolved from a species that is profoundly stupid, as a rule, then perhaps an adaptation/mutation that makes you mistrusting of that species advice is one that tends to propagate. For every "don't use ChatGPT, it's unsafe", there's a "I talk to magical beings who tell me you're not allowed to eat good food and you should mutilate your genitals". Statistically, refusing to learn from other people is probably the best bet.

notavalleyman 3y ago

> Levidow, Levidow & Oberman said in a statement on Thursday that its lawyers “respectfully” disagreed with the court that they had acted in bad faith. “We made a good-faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth,” it said. Seems like they still don't get it. It's not good faith representation of your client, it's not good lawyering to interrogate a chatbot for your case. Even if it could somehow only ever tell the truth. It's not a "good faith mistake" - so I think they still don't understand the problem here. It might be representative of greater society: when people interact with ChatGPT, they treat it as an oracle. I see it all the time on r/chatgpt - people say, "I asked ChatGPT about my long lost grandparent whose life I am researching. Apparently my grandma was a pilot!" Seems like the lawyers are shocked the oracle lied but still think they were right to consult it.

enumjorge 3y ago
The cynic in me thinks they do get it and that their statement is an excuse to save face. It’s easier to blame technology instead of admit they wanted to cut corners.
bitshiftfaced 3y ago
> Seems like they still don't get it. Oh they get it. ChatGPT explained it to them, and then suggested that they take this position because it was the least bad option.
- brewdad 3y ago
  Right. They aren't completely finished with potential punishments. A review by the Bar is likely still ongoing. They can't do anything but pretend it was all done in good faith. Much like a person or company who may end up in a lawsuit can't issue a "real" apology when they screw up lest it be used against them.
  - mirekrusin 3y ago
    I bet a beer they came up with this defense line with a help of gpt.

HumblyTossed 3y ago

Do they just not understand how ChatGPT works? Do they think it's a search engine?

bell-cot 3y ago
"No" and "yes", evidently.
asylteltine 3y ago
[dead]
AHOHA 3y ago
You will be surprised how someone who’s not techie, doesn’t understand tech, heck, I even worked with people who supposedly are tech savvy but lack the basic of the basics. But overall, lawyers and doctors are the least knowledgeable in tech from my experience, I found farmers for example relatively are better, maybe because of a mindset that he/she needs to learn tech to navigate around by themselves, while a lawyer is snob a out it or just hire someone else to do the whole work.
stefan_ 3y ago
Every other HN thread on ChatGPT: artificial general intelligence is near! Isn't this just the same thing as the brain? GPT4 is a game changer! This thread: do they not know what ChatGPT is? Did they not verify the output? How could they believe this? Just maybe when an army of grifters jumps on the next hottest AI thing to advance their careers that kind of bald-faced lying has consequences?
- olddustytrail 3y ago
  Can you genuinely not tell the difference between "this new technology is amazing" and "you should take everything this model says as gospel"? Because if you can't, it's not the fault of "grifters"...

AnIdiotOnTheNet 3y ago

I think many commenters here are unaware of exactly how forehead-slappingly WTF it is that these lawyers did. Even if ChatGPT were replaced with a well respected human legal expert saying exactly the same things, they still cited case law that they couldn't verify the existence of, let alone that the context of the ruling was actually helpful to their cause. Then when a judge told them "hey, we can't find any of these cases, what the hell is going on?" they doubled down. They're basically saying "trust me bro, these rulings totally happened and they totally say we're right".

jeroenhd 3y ago
When criticized about the made up cases they even showed (undated) screenshots of them asking ChatGPT if the cases were real. This is like your local major getting tricked into buying the Eiffel tower using public funds and then defending himself by saying he asked the guy he bought the Eiffel tower from and the guy said he was totally legit.

jeroenhd 3y ago

LegalEagle did a video on this case two weeks ago, digging into it in some more depth: https://youtu.be/oqSYljRYDEM https://youtu.be/oqSYljRYDEM They lied to a judge, doubled down, showed further incompetence by admitting to not knowing how the legal reference they supposedly quoted was even structured, then lied to a judge again trying to get more time to get their story straight, and all it cost them was $5000? And even then they have the guts to say they disagree with the judge? Levidow, Levidow & Oberman must either be run by fools or take the rest of the world for fools. I can understand how the rubber stamping lawyer signing all the paperwork got off easy; he was just signing documents because the lawyer who supposedly did all the work for the case couldn't operate in the court the case was transfered to. The other lawyer, the one who did the AI equivalent from copy pasting from the summary page of Google, should not be deemed fit to practice law.

loeg 3y ago
Also good coverage of the case on Serious Trouble (podcast): https://podcasts.apple.com/us/podcast/the-legal-brief-beyond-the-machine/id1630160928?i=1000615895028 https://podcasts.apple.com/us/podcast/the-legal-brief-beyond...
jcranmer 3y ago
> and all it cost them was $5000? They also have to write written apologies to their client, and all of the judges who were mentioned in their fake cases, with proof filed in the docket. And they are the brunt of a very well-publicized lawsuit. The judge is effectively saying that all that publicity is punishment enough; he doesn't need to do anymore. He doesn't need to refer them to the bar for disbarment--the bar already knows about it. (The judge has no power to disbar them himself, he can only refer to the bar for disbarment.) The biggest surprise is that they didn't get made to pay the defendant's legal fees as well, but that's because defendant didn't ask for any fees. Oh, and the final punishment... they lost the case--they get no chance to redo their opposition to the motion to dismiss. Arguably, their client can also sue them for malpractice, and their malpractice carrier is likely to prefer to settle that rather than actually fight the case.
- 3y ago

jawns 3y ago

Every version of ChatGPT that I have used, going back to the beginning of January 2023, has had a disclaimer appear when you first load up the page that specifically tells you, "May occasionally generate incorrect information." This is why the law firm's contention that they were acting in good faith is laughable. When the tool itself shows a prominent disclaimer that it "May occasionally generate incorrect information," then the law firm can't reasonably argue that "We made a good-faith mistake in failing to believe that a piece of technology could be making up cases out of whole cloth." It was either gross negligence or intentional malfeasance. The most interesting thing to me is why the firm is continuing to try to excuse its actions, rather than taking full responsibility for them. I guess they assume that the classic Shaggy "It Wasn't Me" defense is less likely to lose them business, because they think so little of the clients they're trying to attract that they're wagering they'll buy it.

stblack 3y ago

That's a shockingly low fine.

blindriver 3y ago

There are A LOT of VERY bad lawyers out there. I have a friend who used to be a federal judge for immigration, and he said that immigration lawyers are especially the worst, because families need their services desperately, and these lawyers will take their money and spend NO time on their cases. They show up to court and are figuring stuff out real time. He tore one lawyer a new asshole for doing exactly this for his client and ordered them to come back better prepared. So the idea that some of them will use ChatGPT to do their work doesn't surprise me at all.

ianhawes 3y ago
I work in the immigration space. Part of the problem is that because immigration is technically federal but operates almost entirely in administrative venues (i.e. USCIS and EOIR), immigration lawyers only need to be barred in their home state and do not generally need admission to a relevant federal court bar or appear pro hac vice to represent clients in immigration court or even to make an appearance in filings.
- hgsgm 3y ago
  How is that relevant? Incompetent / criminal lawyers are almost never disbarred in any field.
pclmulqdq 3y ago
It's also not usually a lot of money in comparison to their alternatives. I once had a criminal defense lawyer tell me that they make up the difference with scale and essentially batch-process people.

User23 3y ago

There’s probably a giant opportunity for pro se litigants here. Several possibilities suggest themselves.

jlarocco 3y ago

I wonder if this case was a setup for a lawsuit against OpenAI? Given the marketing and the way Microsoft and Google were tripping over themselves to "catch up", the impression in the media and to the general public is that chat bots are a magic silver bullet for everything. Even a lot of tech people are unclear about exactly how these things work. The case seems pretty clear: "They said it was artificial intelligence, but it wasn't very intelligent to make up cases. Look at the damages it's caused me..."

sh34r 3y ago
“You even said ChatGPT passed the bar exam!” As entertaining as this is to ponder, I don’t think they’re that clever. I don’t think they’d do so much damage to their reputation for such a flimsy case. They’re just incompetent.

lionkor 3y ago

I dont understand - ChatGPT says that it outputs inaccurate information RIGHT BELOW THE PROMPT BOX?! Illiterate lawyers?!

dctoedt 3y ago

In court, good lawyers think of themselves as "officers of the court" — to use a sometimes-forgotten phrase — who are working not just for their clients but also for the judge, to help the judge (and the judge's law clerks) to get it right, with rulings that will stand up on appeal if it comes to that. The good lawyer's job is not just to advocate the client's position — it's also to do "completed staff work" for the court in the process, even acknowledging any weaknesses in the client's case (which builds credibility). As we've seen in the ChatGPT case, judges tend to respond badly when they think a lawyer is trying to put one over on them. And conversely, judges often respond favorably when they perceive that a lawyer is being thorough and intellectually honest, not merely throwing up any bullshit argument that comes to mind. A fond memory from years ago is when a federal judge called me and my opponent — a well-known lawyer in our field — into his chambers to talk about a pending motion that involved an uncertain point of law, which we had extensively briefed. My opponent had not done an especially-great job; my own briefing had acknowledged the weak points in my client's position but also explained why my client should win anyway. The judge said, I don't really know what the answer is here. He then turned to me and said, But I'm more comfortable with your analysis. And he ruled my way – which won the case for my client (a company you might well be a customer of today).

thrw_aus 3y ago

So I guess ChatGPT didn't actually pass the bar exam. I wonder when tech firms will actually be held responsible for false advertisement.

OliverJones 3y ago

And every time this happens from now on, the fine should be doubled. When the fine gets too high for the lazy lawyer to pay, they may instead relinquish their bar admission (lose their license to practice law) for five years. Lawyers admitted to the bar aren't innocents. They're "officers of the court", meaning the legal system relies on them to avoid BS in their citation of other cases and of statutes and laws. It's never been acceptable for a lawyer to sign off on something without reading it and making sure it's right. What if registered professional civil engineers started asking generative AI for sewer-system or bridge designs and then didn't inspect them before approving them and handing them off to the guys with dozers and cement trucks? Not only would people fall into rivers, but the rivers would be polluted. The real world consequences are similarly serious when lawyers make stuff up, or let their interns or word-processing software make stuff up.

karim79 3y ago

It would seem that in the future, people will be less inclined to proof-read their work, or even read it at all prior to submission. I'd wager a guess that this will only get worse over time. LLMs will do more stuff for people - from helping school children to get through their homework, to higher learning assignments and all the way up to the "real" jobs. Along the way people might just place far too much trust in these tools. I'm pretty sure that's something we can all agree on. What I do find striking here though, is the incredible laziness of the lawyers. Not even checking whether or not the cases actually happened? That's just bad lawyering. The AI actually helped the system to call them out on their bad lawyering. That's not "bad things happening because AI" but just plain stupidity.

v3ss0n 3y ago

They just need to become a better prompt engineer. Send them to Andrew Ng course

leoradovey1993 3y ago

[dead]