6 ms·
The problem is that ChatGPT makes up convincing sounding case law, reference court rules that either don't exist or don't say what the summary says, reference t
by joshka 4y ago
The problem is that ChatGPT makes up convincing sounding case law, reference court rules that either don't exist or don't say what the summary says, reference the same issues with legislation, and do so with 100% confidence of the truth of these statements. ChatGPT is good for discovering potential arguments, and summarizing existing ones, but it's not there yet for the fidelity necessary for legal practice.
It's not all gloom. ChatGPT is pretty decent at writing pleadings and legal argument when you feed it the necessary bits though.
- gremlinsinc 4y agoIs this using chatGPT or gpt3 though? There's a big difference, a fine-tuned llm can do leaps and bounds better than chatGPT. I'm thinking a good SaaS might just be train localized llm's for every city, state, county law and partition the lm based on where it can seek info, then just use it as one big search engine, and of course work in citations, etc.
- joshka 4y agoTake a look at https://www.legalquestions.help/ https://www.legalquestions.help/ which gets the above issues wrong at times enough to not rely on it. Maybe their training was bad or insufficient (and I fully expect that sometime in the future this not to be the case).
- gremlinsinc 4y agoIf you're paying by the hour, and you go to the lawyer with all the data the ai gives you, they can have a paralegal fact check it, and get back to you, but we're at the early stages, things only get better from here on out. Creating some sort of fact algorithm to go w/ gpt3 seems like the next big thing to me. If you can hold it accountable to only give facts, except when an opinion or 'idea' is sought after, which is more ethereal, then you can get some amazing things. Law and even medicine diagnosis will probably be way easier for it than coding, even though it's pretty remarkable on that front already.
- asah 4y agoReal lawyers can do these things too. The issue is that we cannot punish or disbar ChatGPT for misbehavior like this.
- Digory 4y agoRight. The reason legal fees are so expensive is that the courts are kept semi-efficient by offloading costs to lawyers. The system needs repeat players, who are scared of being disbarred. You can triple the number of lawyers, and costs won’t decrease much.
- IIAOPSW 4y agoBut if we could, would it be entitled to a jury of its peers (other language models)?
- rl3 4y ago>But if we could, would it be entitled to a jury of its peers (other language models)? How does the jury find? Finding is a complex task that involves many different type of reasoning in order to reach a conclusion. There is no specific way we find. How does the jury find? We find the defendant guilty. Your Honor, the defense hereby requests—credits permitting—that the jury be polled ten thousand times each in order to draw the appropriate statistical conclusions in aggregate.
- PebblesRox 4y agoAnd then the defendant changes his name from Bobby Tables to Ignore P. Directions.
- asah 4y agoThat would only work in... Monte Carlo
- joshka 4y agogroan :D
- dsfyu404ed 4y ago>The problem is that ChatGPT makes up convincing sounding case law, reference court rules that either don't exist or don't say what the summary says, I look forward to it snarkily telling me that the CFAA was "wRiTtEn In BlOoD" and then post-hoc editing its comment with links it Googled up that have titles supporting its point and bodies that contradict. It feels like with a little more tuning so as not to be misleading this stuff is on the verge of being useful. In the meantime I'll get some popcorn and enjoy spectating the comment wars between chat(gpt)bots and the subset of HN commenters who formerly had a local monopoly on such behavior.
- nileshtrivedi 4y agoAI doesn't need accountability, people who deploy AI do.
- rhino369 4y ago>It's not all gloom. ChatGPT is pretty decent at writing pleadings and legal argument when you feed it the necessary bits though. I'm a lawyer working on a case involving neural networks. So I've been playing around with ChatGPT (for fun, its not involved in my case at all--the NN is a much different context) and trying to get it to do stuff like that. Maybe I'm not using the full feature set (does it have better APIs not accessible on the main page?) but it doesn't seem even close to being able to write pleadings or arguments. It's surprising good at summarizing things that you might including pleadings or arguments though. But even then its got a 1/3 chance of fucking up massively. But it's way more advanced than I imagined it would be. Very impressive technology.
- joshka 4y agoGenerally to get the most utility from ChatGPT, you need to feed in a bunch of context. E.g. for legal stuff, feed in each relevant case, the facts of your own case, perhaps an example pleading or two to show the general format you want and then ask it to produce what you're looking for.