5 ms·

> “I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic,” Mr.

> “I simply had no idea that ChatGPT was capable of fabricating entire case citations or judicial opinions, especially in a manner that appeared authentic,” Mr. Schwartz said in a declaration filed on Tuesday with the judge, P. Kevin Castel of Federal District Court. Your whole occupation literally depends on not inventing bullshit on the spot, and yet you use technology that is marked, by its creator, with big letters, as a bullshit inventer. What can go wrong?

elforce002 3y ago
This. Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?? I'm not for heavy regulations myself but for Medicine, Law, Civil engineering we can make a case on why we need regulations in these areas.
- sidewndr46 3y ago
  Well, that does exist: https://reillytop10.com/previous-lists/2018-list/sentencing-software/ https://reillytop10.com/previous-lists/2018-list/sentencing-...
- engineer_22 3y ago
  Agreed, and happily they're already regulated. Practitioners of engineering and medicine can be criminally liable for negligence. The government also regulates the profession by granting licenses and preventing people from practicing in these fields without a license. Law, well, it's self regulated...
- Someone1234 3y ago
  > Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?? This was a civil lawsuit wherein a passenger got injured on an aircraft. I'm not trying to lessen what occurred, just trying to accurately characterize the risks associated with this professional misconduct.
- tikhonj 3y ago
  From what I've read about the US justice system, you don't have to imagine very hard—it just happens through institutionally supported channels like pushing people to plea bargains and underfunding public defenders' offices.
- giantg2 3y ago
  "Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?" Interesting question. Most people have a lot of bias that wouldn't allow themself to see that the person they locked up might be innocent. You'd have almost nobody working as a judge, lawyer, cop, etc if the cognitive dissonance were allowed to creep in. In the vast majority of cases they only leveled justice against the person because they thought it was right.
JohnFen 3y ago
That Mr. Schwartz offered this as a kind of defense makes me seriously question his ability as a lawyer. Even if he had a very high degree of confidence in the tool, he has to know that you fact-check anyway. Just as you would if a human produced it.
- akiselev 3y ago
  It was a complete abdication of professional responsibility to his client. Doesn’t matter whether he pulled the citations from a paralegal, ChatGPT, or Google - it’s his job to check and validate before submitting anything to the court Instead “aw shucks boys will be boys”
  - colpabar 3y ago
    > Instead “aw shucks boys will be boys” What do you mean? He seems to be facing potential legal trouble, and on top of that he and his firm will forever be known as the dumbasses who submitted fake court decisions in a real case.
    - ender341341 3y ago
      I think akiselev's claim is that the lawyer is trying to claim that, not that that's how it's being treated.
      - akiselev 3y ago
        Correct. It's an esoteric reference to Lisa Simpson's motion to declare writ of boys will be boys in defense of her brother [1] [1] https://www.youtube.com/watch?v=-QsR2CZ1fzE&t=87 https://www.youtube.com/watch?v=-QsR2CZ1fzE&t=87
        ender341341 3y ago
        That clip was what I thought of when reading the comment.
- 3y ago
engineer_22 3y ago
Agreed. Lawyers are expected to be detail oriented and highly engaged with the factual and theoretical basis for their arguments. To be citing cases that you are not familiar with, by itself that's very sloppy.
slg 3y ago
This is just how a lot of normal people use technology. They aren't going to read everything on the page. They will read the bare minimum needed to work the software and sometimes they won't even read that. I would bet this guy isn't out of the ordinary and that a large percentage of ChatGPT users don't know the system's propensity for just making up authentic looking results.
ww520 3y ago
He’s lying. In the transcript, he asked ChatGPT, “are the cases real? … are you sure?” That means in his mind he knew ChatGPT is capable of making things up and wanted to verify. He’s just lazy and didn’t do a simple external verification. May he knew that if he did an external verification and the cites turned out to be false, he had to rewrite the whole brief.
giantg2 3y ago
"Your whole occupation literally depends on not inventing bullshit on the spot," Doesn't surprise me in the least. Incompetence and short cuts are common in the legal profession from my experience. It also seems their jobs fo involve quite a bit of bullshit, or at least using real info in a bullshit way (it's what the judge/jury perceived, not necessarily reality or the truth).
- danielmarkbruce 3y ago
  Every profession...

elforce002 3y ago

This. Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?? I'm not for heavy regulations myself but for Medicine, Law, Civil engineering we can make a case on why we need regulations in these areas.

sidewndr46 3y ago
Well, that does exist: https://reillytop10.com/previous-lists/2018-list/sentencing-software/ https://reillytop10.com/previous-lists/2018-list/sentencing-...
engineer_22 3y ago
Agreed, and happily they're already regulated. Practitioners of engineering and medicine can be criminally liable for negligence. The government also regulates the profession by granting licenses and preventing people from practicing in these fields without a license. Law, well, it's self regulated...
Someone1234 3y ago
> Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?? This was a civil lawsuit wherein a passenger got injured on an aircraft. I'm not trying to lessen what occurred, just trying to accurately characterize the risks associated with this professional misconduct.
tikhonj 3y ago
From what I've read about the US justice system, you don't have to imagine very hard—it just happens through institutionally supported channels like pushing people to plea bargains and underfunding public defenders' offices.
giantg2 3y ago
"Can you imagine sending someone to jail or leaving someone to walk free just because you were too lazy to actually do your job?" Interesting question. Most people have a lot of bias that wouldn't allow themself to see that the person they locked up might be innocent. You'd have almost nobody working as a judge, lawyer, cop, etc if the cognitive dissonance were allowed to creep in. In the vast majority of cases they only leveled justice against the person because they thought it was right.

JohnFen 3y ago

That Mr. Schwartz offered this as a kind of defense makes me seriously question his ability as a lawyer. Even if he had a very high degree of confidence in the tool, he has to know that you fact-check anyway. Just as you would if a human produced it.

akiselev 3y ago
It was a complete abdication of professional responsibility to his client. Doesn’t matter whether he pulled the citations from a paralegal, ChatGPT, or Google - it’s his job to check and validate before submitting anything to the court Instead “aw shucks boys will be boys”
- colpabar 3y ago
  > Instead “aw shucks boys will be boys” What do you mean? He seems to be facing potential legal trouble, and on top of that he and his firm will forever be known as the dumbasses who submitted fake court decisions in a real case.
  - ender341341 3y ago
    I think akiselev's claim is that the lawyer is trying to claim that, not that that's how it's being treated.
    - akiselev 3y ago
      Correct. It's an esoteric reference to Lisa Simpson's motion to declare writ of boys will be boys in defense of her brother [1] [1] https://www.youtube.com/watch?v=-QsR2CZ1fzE&t=87 https://www.youtube.com/watch?v=-QsR2CZ1fzE&t=87
      - ender341341 3y ago
        That clip was what I thought of when reading the comment.
3y ago

engineer_22 3y ago

Agreed. Lawyers are expected to be detail oriented and highly engaged with the factual and theoretical basis for their arguments. To be citing cases that you are not familiar with, by itself that's very sloppy.

slg 3y ago

This is just how a lot of normal people use technology. They aren't going to read everything on the page. They will read the bare minimum needed to work the software and sometimes they won't even read that. I would bet this guy isn't out of the ordinary and that a large percentage of ChatGPT users don't know the system's propensity for just making up authentic looking results.

ww520 3y ago

He’s lying. In the transcript, he asked ChatGPT, “are the cases real? … are you sure?” That means in his mind he knew ChatGPT is capable of making things up and wanted to verify. He’s just lazy and didn’t do a simple external verification. May he knew that if he did an external verification and the cites turned out to be false, he had to rewrite the whole brief.

giantg2 3y ago

"Your whole occupation literally depends on not inventing bullshit on the spot," Doesn't surprise me in the least. Incompetence and short cuts are common in the legal profession from my experience. It also seems their jobs fo involve quite a bit of bullshit, or at least using real info in a bullshit way (it's what the judge/jury perceived, not necessarily reality or the truth).

danielmarkbruce 3y ago
Every profession...