4 ms·

It had to be shot down harshly because there are some premises to a courtroom proceeding that aren't met by an AI as we currently have. One of those is that th

It had to be shot down harshly because there are some premises to a courtroom proceeding that aren't met by an AI as we currently have. One of those is that the lawyer arguing a case is properly credentialed and has been admitted to the bar, and is a professional subject to malpractice standards, who can be held responsible for their performance. An AI spitting out statistically likely responses can't be considered an actual party to the proceedings in that sense. If a lawyer cites a non-existent precedent, they can make their apologies to the court or be sanctioned. If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? Someone standing in court, repeating what they hear through an earpiece, is literally committing a fraud on the court by presenting themselves as a credentialled attorney. The stunt of "haha, it was really just chatGPT!" would have had severe legal consequences for everyone involved. The harsh response saved DoNotPay from itself.

john_the_writer 4y ago
I don't understand the "cites a non-existent precedent" bit. Presumably the AI would have a database of a pile of precedent. It wouldn't make up cites. It would have "knowledge" of so much precedence, it could likely find something to win either side of the argument.
- fatbird 4y ago
  I think you're misunderstanding how the model works. It predicts next tokens based on past tokens and the LLM trained on large bodies of text. It doesn't have an underlying database of "factual" elements it incorporates or searches, and its output doesn't have an underlying semantic structure that can be verified or reasoned about. The entirety of the quality of its output can only be judged by whether it "sounds" like the rest of the text on which it was trained. I think making the connection between the predictive output and an underlying representation of reality is the next great step, but until that happens, chatGPT's output is just amazing mimicry of human language.
freedomben 4y ago
> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building all your own facts. I used it to generate some ffmpeg commands. I had to verify all the flags myself, but it was like 5 minutes of work compared to probably hours it would have taken me to figure them all out on my own.
- thombat 4y ago
  But when appearing in court you're in real-time: you can't take 5 minutes to validate the AI output before passing it on. You can do that for your opening statements but once faced with the judge's rulings or cross-examination you'll be in the weeds.
  - freedomben 4y ago
    Yeah that's fair, although if it was AI-assisted lawyer then presumably you'd have done the research ahead of time. But, for spontaneous stuff, you're totally right. My original statement was thinking about it as a "prep time" exercise, but spontaneous stuff would appear in court. Although, the human lawyer (who should still be simiarly prepared for court) would be there to handle those, possibly with some quick assistance.
    - dragonwriter 4y ago
      > if it was AI-assisted lawyer If it was AI-assisted lawyer, it would be a whole different discussion. Aside from requiring a live feed of interactions to a remote system and other technical details, “lawyers using supportive tools while exercising their own judgement on behalf of their client” isn’t controversial the way marketing an automated system as, or as a substitute for, legal counsel and representation is.
- gamblor956 4y ago

john_the_writer 4y ago

I don't understand the "cites a non-existent precedent" bit. Presumably the AI would have a database of a pile of precedent. It wouldn't make up cites. It would have "knowledge" of so much precedence, it could likely find something to win either side of the argument.

fatbird 4y ago
I think you're misunderstanding how the model works. It predicts next tokens based on past tokens and the LLM trained on large bodies of text. It doesn't have an underlying database of "factual" elements it incorporates or searches, and its output doesn't have an underlying semantic structure that can be verified or reasoned about. The entirety of the quality of its output can only be judged by whether it "sounds" like the rest of the text on which it was trained. I think making the connection between the predictive output and an underlying representation of reality is the next great step, but until that happens, chatGPT's output is just amazing mimicry of human language.

freedomben 4y ago

> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI? IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building all your own facts. I used it to generate some ffmpeg commands. I had to verify all the flags myself, but it was like 5 minutes of work compared to probably hours it would have taken me to figure them all out on my own.

thombat 4y ago
But when appearing in court you're in real-time: you can't take 5 minutes to validate the AI output before passing it on. You can do that for your opening statements but once faced with the judge's rulings or cross-examination you'll be in the weeds.
- freedomben 4y ago
  Yeah that's fair, although if it was AI-assisted lawyer then presumably you'd have done the research ahead of time. But, for spontaneous stuff, you're totally right. My original statement was thinking about it as a "prep time" exercise, but spontaneous stuff would appear in court. Although, the human lawyer (who should still be simiarly prepared for court) would be there to handle those, possibly with some quick assistance.
  - dragonwriter 4y ago
    > if it was AI-assisted lawyer If it was AI-assisted lawyer, it would be a whole different discussion. Aside from requiring a live feed of interactions to a remote system and other technical details, “lawyers using supportive tools while exercising their own judgement on behalf of their client” isn’t controversial the way marketing an automated system as, or as a substitute for, legal counsel and representation is.
gamblor956 4y ago