3 ms·
> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model agai
by freedomben 4y ago
> If the AI cites a non-existent precedent, there's literally no way to incorporate that error back into the AI because there's no factual underlying model against which to check the AI's output--unless you had an actual lawyer checking it, in which case, what's the point of the AI?
IANAL, but I would bet the level of effort to fact check an AI's output would be orders of magnitude lower than researching and building all your own facts.
I used it to generate some ffmpeg commands. I had to verify all the flags myself, but it was like 5 minutes of work compared to probably hours it would have taken me to figure them all out on my own.
- thombat 4y agoBut when appearing in court you're in real-time: you can't take 5 minutes to validate the AI output before passing it on. You can do that for your opening statements but once faced with the judge's rulings or cross-examination you'll be in the weeds.
- freedomben 4y agoYeah that's fair, although if it was AI-assisted lawyer then presumably you'd have done the research ahead of time. But, for spontaneous stuff, you're totally right. My original statement was thinking about it as a "prep time" exercise, but spontaneous stuff would appear in court. Although, the human lawyer (who should still be simiarly prepared for court) would be there to handle those, possibly with some quick assistance.
- dragonwriter 4y ago> if it was AI-assisted lawyer If it was AI-assisted lawyer, it would be a whole different discussion. Aside from requiring a live feed of interactions to a remote system and other technical details, “lawyers using supportive tools while exercising their own judgement on behalf of their client” isn’t controversial the way marketing an automated system as, or as a substitute for, legal counsel and representation is.
- gamblor956 4y agoYou would lose that bet. Fact-checking nonsensical output would take a lot longer than researching a single body of law, which you can generally do by just looking up a recent case on the matter. You don't need to check every cite; that will have been done for you by the lawyers and judges involved in that case. But checking every cite in an AI's output: many of those citations won't exist, and for the ones that do, you'll need to closely read all of them to confirm that they say what the AI claims they say, or are even within the ballpark of what the AI claims they say.
- bluGill 4y agoFact checking an AI is still massively easier than finding and reading all the precedent yourself. Real lawyers of course already know the important precedent in the areas they deal in, and they still have teams behind the scene to search out more that might apply, and then only read the ones the team says look important. Of course there could be a difference between an reading all the cases an AI says are important and actually finding the important cases including ones the AI didn't point you at. However this is not what the bet was about.
- dragonwriter 4y ago> Fact checking an AI is still massively easier than finding and reading all the precedent yourself. Actually fact-checking an AI requires finding and reading all the precedent yourself to verify that the AI has both cited accurately and not missed contradictory precedent that is more relevant (whether newer, from a higher court, or more specifically on-point.) If it has got an established track record, just as with a human assistant, you can make an informed decision about what corners you can afford to cut on that, but then you aren't really fact-checking it. OTOH, an AI properly trained on one of the existing human-curated and annotated databases linking case law to issues and tracking which cases apply, overrule, or modify holdings from others might be extremely impressive—but those are likely to be expensive products tied to existing offerings from Westlaw, LexisNexis, etc.
- john_the_writer 4y ago