14 ms·

This is one of the domains I'm very very excited about for LLMs to help me with. In 5-10 years (even though this research paper makes me feel its already here),

This is one of the domains I'm very very excited about for LLMs to help me with. In 5-10 years (even though this research paper makes me feel its already here), I would feel very confident chatting for a few hours with a "lawyer" LLM that has access to all my relevant taxes/medical/insurance/marriage documents and would be able to give me specialized advice and information without billing me $500 an hour. A wave of (better) legally informed common-person is coming, and I couldn't be more excited!

OldOneEye 3y ago
I wouldn't blindly trust what the LLM says, but I take it that it would be mostly right, and that would give me at the very least explorable vocabulary that I can expand on my own, or keep grilling it about. I've already used some LLMs to ask questions about licenses and legal consequences for software related matters, and it gave me a base, without having to involve a very expensive professional into it for what are mostly questions for hobby things I'm doing. If there was a significant amount of money involved in the decision, though, I will of course use the services of a professional. These are the kinds of topics you can't be "mostly right".
- nprateem 3y ago
  The only problems are it could be convincingly wrong about anything it tells you and isn't liable for its mistakes.
  - sonofaragorn 3y ago
    What if they were liable? Say the company that offers the LLM lawyer is liable. Would that make this feasible? In terms of being convincingly wrong, it's not like lawyers never make mistakes...
    - giantg2 3y ago
      "What if they were liable?" They'd be sued out of existence. "In terms of being convincingly wrong, it's not like lawyers never make mistakes..." They have malpractice insurance, they can potentially defend their position if later sued, and most importantly they have the benefit of appeal to authority image/perception.
      - AnimalMuppet 3y ago
        All right, what if legal GPTs had to carry malpractice insurance? Either they give good advice, or the insurance rates will drive them out of business. I guess you'd have to have some way of knowing that the "malpractice insurance ID" that the GPT gave you at the start of the session was in fact valid, and with an insurance company that had the resources to actually cover if needed...
        kulikalov 3y ago
        It's funny how any conversation ends with this question unanswered.
        YetAnotherNick 3y ago
        Weirdly HN is full of anti AI people who just refuses to discuss the point that is being discussed and goes into all the same argument of wrong answer that they got some time. And then they present anecdotal evidence as truth, while there is no clear evidence if AI lawyer has more or less chance to be wrong than human. Surely AI could remember more and has been shown to clear bar examination.
        giantg2 3y ago
        "while there is no clear evidence if AI lawyer has more or less chance to be wrong than human." In the tests they are shown to be pretty close. The point I made wasn't about more mistakes, but about other factors influencing liability and how it would be worse for AI than humans at this point.
    - naniwaduni 3y ago
      You'd require them to carry liability insurance (this is usually true for meat lawyers as well), which basically punts the problem up to "how good do they have to be to convince an insurer to offer them an appropriate amount of insurance at a price that leaves the service economically viable?"
      - kulikalov 3y ago
        Given orders of magnitude better cost efficiency, they will have plenty of funds to lure in any insurance firm in existence. And then replace insurance firms too.
  - engineer_22 3y ago
    This is an area for further development and thought... If a LLM can pass the bar, and has a corpus of legal work instantly accessible, what prevents the deployment of the LLM (or other AI structure) to provide legitimate legal services? If the AI is providing legal services, how do we assign responsibility for the work (to the AI, or to its owner)? How to insure the work for Errors and Omissions? More practically, if willing to take on responsibility for yourself, is the use of AI going to save you money?
    - AnimalMuppet 3y ago
      A human that screws up either too often or too spectacularly can be disbarred, even if they passed the bar. They can also be sued. If a GPT screws up, it could in theory be disbarred. But you can't sue it for damages, and you can't tell whether the same model under a different name is the next legal GPT you consult.
      - engineer_22 3y ago
        Agreed - which is why this is an area that needs more thought and development
    - nprateem 3y ago
      Re your first point: it's not conscious. It has no understanding. It's perfectly possible the model could successfully answer an exam question but fail to reach the same or similar conclusion when it has to reason it's own way there based on information provided.
      - mvdtnz 3y ago
        Careful, there are plenty of True Believers on this website who really think that these "guess the next word" machines really do have consciousness and understanding.
        lewhoo 3y ago
        I incline towards you on the subject but if you call it guessing you open yourself up to all sorts of rebuttals.
  - boplicity 3y ago
    The obvious intermediate step is that you add an actual expert into the workflow, in terms of using LLMs for this purpose. Basically, add a "validate" step. So, you'd first chat with the LLM, create conclusions, then vet those conclusions with an expert specially trained to be skeptical of LLM generated content. I would be shocked if there aren't law agencies that aren't already doing something exactly like this.
    - freejazz 3y ago
      Ah, so have the lawyer do everything the GPT did so the lawyer can be sure the GPT didn't fuck up.
  - deleted 3y ago
    [deleted]
- thallium205 3y ago
  I wouldn't blindly trust what a lawyer says either so there's no difference there.
  - brk 3y ago
    Sure, but you have a lot less personal risk following advice from a lawyer vs. advice from an LLM.
    - toomuchtodo 3y ago
      When your GPT is wrong, you will be laughed out of the room and sanctioned. When your attorney is wrong, you get to point at the attorney and show a good faith effort was made. Hacks are fun, just keep in mind the domain you're operating in.
      - giantg2 3y ago
        "When your attorney is wrong, you get to point at the attorney and show a good faith effort was made." And possibly sue their insurance to correct their mistakes.
        deleted 3y ago
        [deleted]
        Scoundreller 3y ago
        But you’ll have to find a lawyer that specializes in suing lawyers and their own malpractice plans. Maybe that’s where legal AI will find the most demand.
        kulikalov 3y ago
        Can't a tech firm running a "legal gpt" have an insurance?
        toomuchtodo 3y ago
        Do they have a license to practice law?
- mannykannot 3y ago
  I like the term "explorable vocabulary." I can see using LLMs to get an idea of what the relevant issues are before I approach a professional, without assuming that any particular claim in the model's responses is correct.
- chaxor 3y ago
  I don't understand how everyone keeps making this mistake over and over. They explicitly just said "in 5-10 years". So many people continually use arguments that revolve around 'I used it once and it wasn't the best and/or me things up', and imply that this will always be the case. There are many solutions already for knowledge editing, there are many solutions for improving performance, and there will very likely continue to be many improvements across the board for this. It took ~5 years from when people in the NLP literature noticed BERT and knew the powerful applications that were coming, until the public at large was aware of the developments via ChatGPT. It may take another 5 before the public sees the developments happening now in the literature hit something in a companies web UI.
  - mjr00 3y ago
    on the other hand the rate of change isn't constant and there isn't a guarantee that the incredible progress in the past ~2 years in the LLM/diffusion/"AI" space will continue. As an example, take computer gaming graphics; compare the evolution between Wolfenstein 3D (1992) and Quake 3 Arena (1999), which is an absolute quantum leap. Now compare Resident Evil 7 (2017) and Alan Wake 2 (2023) and it's an improvement but nowhere near the same scale. We've already seen a fair bit of stagnation in the past year as ChatGPT gets progressively worse as the company is more focusing on neutering results to limit its exposure to legal liability.
    - GaggiX 3y ago
      >ChatGPT gets progressively worse https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar..., In blinded human comparisons, newer models perform better than older ones.
      - mvdtnz 3y ago
        That website doesn't load for me but anyone who uses ChatGPT semi regularly can see that it's getting steadily worse if you ever ask for anything that begins to border risque. It has even refused to provide me with things like bolt torque specs because of risk.
- coffeebeqn 3y ago
  I wonder could GPTs come up with legal loopholes. Like they are expected to come up with security vulnerabilities
XCSme 3y ago
Or a LLM that helps you spend less. Imagine a LLM that goes over all your spending, knows all the current laws, benefits, organizations, promotional campaigns, and suggests (or even executes) things like changing electricity provider, insurance provider, buying stuff in bulk from a different shop that you get for 4x the price at your local store, etc.
- OldOneEye 3y ago
  I love this idea. It would be incredibly useful! I feel LLMs are great at suggestions that you follow up yourself (if only for sanity checking, but nothing you wouldn't do with a human too).
- p-e-w 3y ago
  That would not be in the interest of anyone with any real power, so you're going to see tanks on the streets before it happens.
Solvency 3y ago
This does not make sense to me. ChatGPT is completely nerfed to the point where it's either been conditioned or trained to provide absolutely zero concrete responses to anything. All it does is provide the most baseline, generic possible response followed by some throwaway recommendation to seek the advice of an actual expert.
- frankfrank13 3y ago
  The way to get around this is to have it "quote" or at least try to quote from input documents. Which is why RAG became so popular. Sure, it won't right you a contract, but it will read one back to you if you've provided one in your prompt. In my experience, this does not get you close to what the top-level comment is describing. But it gets around the "nerfing" you describe
- baobabKoodaa 3y ago
  It's very easy to get ChatGPT to provide legal advice based on information fed in the prompt. OpenAI is not censoring legal advice anywhere near as hard as they are censoring politics or naughty talk.
  - freejazz 3y ago
    That's because its just advice, not legal advice. Legal advice is something you get from an attorney that represents you. It creates a liability relationship between you and the attorney for the legal advice they did provide.
    - baobabKoodaa 3y ago
      Sure, we can call it "advice" instead of "legal advice" or we can even call it other names, like "potato", if that's what you want. My point is that potato not censored.
      - freejazz 3y ago
        Feel free to miss the point as much as you want. You can call it a baked potato then!
frankfrank13 3y ago
I think a lot of startups are working on exactly what you are describing, and honestly, I wouldn't hold my breath. Everyone is still bound by token limits and the two best approaches for getting around them are RAG and Knowledge-Graphs, both of which could get you close to what you describe but not close enough to be useful (IMO).
baobabKoodaa 3y ago
We are literally building this today! Our core business is legal document generation (rule based logic, no AI). Since we already have the users' legal documents available to us as a result of our core business, we are perfectly positioned to build supplementary AI chat features related to legal documents. We recently deployed a product recommendation AI to prod (partially rule based, but personalized recommendation texts generated by GPT-4). We are currently building AI chat features to help users understand different legal documents and our services. We're intending to replace the first level of customer support with this AI chat (and before you get upset, know that the first level of customer support is currently a very bad rule-based AI). Main website in Finnish: https://aatos.app https://aatos.app (also some services for SE and DK, plus we recently opened UK with just a e-sign service)
- baobabKoodaa 3y ago
  Here's an example of what our product recommendations look like: Given your ownership in a company and real estate, a lasting power of attorney is a prudent step. This allows you to appoint PARTNER_NAME or another trusted individual to manage your business and property affairs in the event of incapacitation. Additionally, it can also provide tax benefits by allowing tax-free gifts to your children, helping to avoid unnecessary inheritance taxes and secure the financial future of your large family.
- Closi 3y ago
  > Since we already have the users' legal documents available to us as a result of our core business, we are perfectly positioned to build supplementary AI chat features related to legal documents. Uhh... What are the privacy implications here?!
  - baobabKoodaa 3y ago
    If you look at the example I posted of our product recommendations, you will see that the GPT-4 generated text contains "PARTNER_NAME" instead of actual partner name. That's because we've created anonymized dataset from users in such a way that it's literally impossible for OpenAI to deanonymize users. Of course the same can not be done if we want to provide a service where users can, for example, chat with their legal documents. In that case we will have to send some private details to OpenAI. Not sure how that will pan out (what details we decide to send and what we decide not to send). In any case, all startups today are created on top of a mountain of cloud services. Any one of those services can leak private user data as a result of outsider hack or insider attack or accident. OpenAI is just one more cloud service on top of the mountain.
nostromo 3y ago
And not just legal either. I uploaded all of my bloodwork tests and my 23andme data to Chat GPT and it was better at analyzing it than my doctor was.
- slingnow 3y ago
  This is a really interesting use case for me. I've been envisioning a specially trained LLM that can give useful advice or insights that your average PCP might gloss over or not have the time to investigate. Did you do anything special to achieve this? What were the results like?
- p-e-w 3y ago
  Yup. Because the doctor doesn't have time and doesn't give a fuck. LLMs don't have to compete against the cutting edge of human professional knowledge. They only have to compete against the disinterested, arrogant, greedy, and overworked professionals that are actually available to people in practice. No wonder they're winning.

OldOneEye 3y ago

I wouldn't blindly trust what the LLM says, but I take it that it would be mostly right, and that would give me at the very least explorable vocabulary that I can expand on my own, or keep grilling it about. I've already used some LLMs to ask questions about licenses and legal consequences for software related matters, and it gave me a base, without having to involve a very expensive professional into it for what are mostly questions for hobby things I'm doing. If there was a significant amount of money involved in the decision, though, I will of course use the services of a professional. These are the kinds of topics you can't be "mostly right".

nprateem 3y ago
The only problems are it could be convincingly wrong about anything it tells you and isn't liable for its mistakes.
- sonofaragorn 3y ago
  What if they were liable? Say the company that offers the LLM lawyer is liable. Would that make this feasible? In terms of being convincingly wrong, it's not like lawyers never make mistakes...
  - giantg2 3y ago
    "What if they were liable?" They'd be sued out of existence. "In terms of being convincingly wrong, it's not like lawyers never make mistakes..." They have malpractice insurance, they can potentially defend their position if later sued, and most importantly they have the benefit of appeal to authority image/perception.
    - AnimalMuppet 3y ago
      All right, what if legal GPTs had to carry malpractice insurance? Either they give good advice, or the insurance rates will drive them out of business. I guess you'd have to have some way of knowing that the "malpractice insurance ID" that the GPT gave you at the start of the session was in fact valid, and with an insurance company that had the resources to actually cover if needed...
      - kulikalov 3y ago
        It's funny how any conversation ends with this question unanswered.
        YetAnotherNick 3y ago
        Weirdly HN is full of anti AI people who just refuses to discuss the point that is being discussed and goes into all the same argument of wrong answer that they got some time. And then they present anecdotal evidence as truth, while there is no clear evidence if AI lawyer has more or less chance to be wrong than human. Surely AI could remember more and has been shown to clear bar examination.
        giantg2 3y ago
        "while there is no clear evidence if AI lawyer has more or less chance to be wrong than human." In the tests they are shown to be pretty close. The point I made wasn't about more mistakes, but about other factors influencing liability and how it would be worse for AI than humans at this point.
  - naniwaduni 3y ago
    You'd require them to carry liability insurance (this is usually true for meat lawyers as well), which basically punts the problem up to "how good do they have to be to convince an insurer to offer them an appropriate amount of insurance at a price that leaves the service economically viable?"
    - kulikalov 3y ago
      Given orders of magnitude better cost efficiency, they will have plenty of funds to lure in any insurance firm in existence. And then replace insurance firms too.
- engineer_22 3y ago
  This is an area for further development and thought... If a LLM can pass the bar, and has a corpus of legal work instantly accessible, what prevents the deployment of the LLM (or other AI structure) to provide legitimate legal services? If the AI is providing legal services, how do we assign responsibility for the work (to the AI, or to its owner)? How to insure the work for Errors and Omissions? More practically, if willing to take on responsibility for yourself, is the use of AI going to save you money?
  - AnimalMuppet 3y ago
    A human that screws up either too often or too spectacularly can be disbarred, even if they passed the bar. They can also be sued. If a GPT screws up, it could in theory be disbarred. But you can't sue it for damages, and you can't tell whether the same model under a different name is the next legal GPT you consult.
    - engineer_22 3y ago
      Agreed - which is why this is an area that needs more thought and development
  - nprateem 3y ago
    Re your first point: it's not conscious. It has no understanding. It's perfectly possible the model could successfully answer an exam question but fail to reach the same or similar conclusion when it has to reason it's own way there based on information provided.
    - mvdtnz 3y ago
      Careful, there are plenty of True Believers on this website who really think that these "guess the next word" machines really do have consciousness and understanding.
      - lewhoo 3y ago
        I incline towards you on the subject but if you call it guessing you open yourself up to all sorts of rebuttals.
- boplicity 3y ago
  The obvious intermediate step is that you add an actual expert into the workflow, in terms of using LLMs for this purpose. Basically, add a "validate" step. So, you'd first chat with the LLM, create conclusions, then vet those conclusions with an expert specially trained to be skeptical of LLM generated content. I would be shocked if there aren't law agencies that aren't already doing something exactly like this.
  - freejazz 3y ago
    Ah, so have the lawyer do everything the GPT did so the lawyer can be sure the GPT didn't fuck up.
- deleted 3y ago
  [deleted]
thallium205 3y ago
I wouldn't blindly trust what a lawyer says either so there's no difference there.
- brk 3y ago
  Sure, but you have a lot less personal risk following advice from a lawyer vs. advice from an LLM.
  - toomuchtodo 3y ago
    When your GPT is wrong, you will be laughed out of the room and sanctioned. When your attorney is wrong, you get to point at the attorney and show a good faith effort was made. Hacks are fun, just keep in mind the domain you're operating in.
    - giantg2 3y ago
      "When your attorney is wrong, you get to point at the attorney and show a good faith effort was made." And possibly sue their insurance to correct their mistakes.
      - deleted 3y ago
        [deleted]
      - Scoundreller 3y ago
        But you’ll have to find a lawyer that specializes in suing lawyers and their own malpractice plans. Maybe that’s where legal AI will find the most demand.
      - kulikalov 3y ago
        Can't a tech firm running a "legal gpt" have an insurance?
        toomuchtodo 3y ago
        Do they have a license to practice law?
mannykannot 3y ago
I like the term "explorable vocabulary." I can see using LLMs to get an idea of what the relevant issues are before I approach a professional, without assuming that any particular claim in the model's responses is correct.
chaxor 3y ago
I don't understand how everyone keeps making this mistake over and over. They explicitly just said "in 5-10 years". So many people continually use arguments that revolve around 'I used it once and it wasn't the best and/or me things up', and imply that this will always be the case. There are many solutions already for knowledge editing, there are many solutions for improving performance, and there will very likely continue to be many improvements across the board for this. It took ~5 years from when people in the NLP literature noticed BERT and knew the powerful applications that were coming, until the public at large was aware of the developments via ChatGPT. It may take another 5 before the public sees the developments happening now in the literature hit something in a companies web UI.
- mjr00 3y ago
  on the other hand the rate of change isn't constant and there isn't a guarantee that the incredible progress in the past ~2 years in the LLM/diffusion/"AI" space will continue. As an example, take computer gaming graphics; compare the evolution between Wolfenstein 3D (1992) and Quake 3 Arena (1999), which is an absolute quantum leap. Now compare Resident Evil 7 (2017) and Alan Wake 2 (2023) and it's an improvement but nowhere near the same scale. We've already seen a fair bit of stagnation in the past year as ChatGPT gets progressively worse as the company is more focusing on neutering results to limit its exposure to legal liability.
  - GaggiX 3y ago
    >ChatGPT gets progressively worse https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar..., In blinded human comparisons, newer models perform better than older ones.
    - mvdtnz 3y ago
      That website doesn't load for me but anyone who uses ChatGPT semi regularly can see that it's getting steadily worse if you ever ask for anything that begins to border risque. It has even refused to provide me with things like bolt torque specs because of risk.
coffeebeqn 3y ago
I wonder could GPTs come up with legal loopholes. Like they are expected to come up with security vulnerabilities

XCSme 3y ago

Or a LLM that helps you spend less. Imagine a LLM that goes over all your spending, knows all the current laws, benefits, organizations, promotional campaigns, and suggests (or even executes) things like changing electricity provider, insurance provider, buying stuff in bulk from a different shop that you get for 4x the price at your local store, etc.

OldOneEye 3y ago
I love this idea. It would be incredibly useful! I feel LLMs are great at suggestions that you follow up yourself (if only for sanity checking, but nothing you wouldn't do with a human too).
p-e-w 3y ago
That would not be in the interest of anyone with any real power, so you're going to see tanks on the streets before it happens.

Solvency 3y ago

This does not make sense to me. ChatGPT is completely nerfed to the point where it's either been conditioned or trained to provide absolutely zero concrete responses to anything. All it does is provide the most baseline, generic possible response followed by some throwaway recommendation to seek the advice of an actual expert.

frankfrank13 3y ago
The way to get around this is to have it "quote" or at least try to quote from input documents. Which is why RAG became so popular. Sure, it won't right you a contract, but it will read one back to you if you've provided one in your prompt. In my experience, this does not get you close to what the top-level comment is describing. But it gets around the "nerfing" you describe
baobabKoodaa 3y ago
It's very easy to get ChatGPT to provide legal advice based on information fed in the prompt. OpenAI is not censoring legal advice anywhere near as hard as they are censoring politics or naughty talk.
- freejazz 3y ago
  That's because its just advice, not legal advice. Legal advice is something you get from an attorney that represents you. It creates a liability relationship between you and the attorney for the legal advice they did provide.
  - baobabKoodaa 3y ago
    Sure, we can call it "advice" instead of "legal advice" or we can even call it other names, like "potato", if that's what you want. My point is that potato not censored.
    - freejazz 3y ago
      Feel free to miss the point as much as you want. You can call it a baked potato then!

frankfrank13 3y ago

I think a lot of startups are working on exactly what you are describing, and honestly, I wouldn't hold my breath. Everyone is still bound by token limits and the two best approaches for getting around them are RAG and Knowledge-Graphs, both of which could get you close to what you describe but not close enough to be useful (IMO).

baobabKoodaa 3y ago

We are literally building this today! Our core business is legal document generation (rule based logic, no AI). Since we already have the users' legal documents available to us as a result of our core business, we are perfectly positioned to build supplementary AI chat features related to legal documents. We recently deployed a product recommendation AI to prod (partially rule based, but personalized recommendation texts generated by GPT-4). We are currently building AI chat features to help users understand different legal documents and our services. We're intending to replace the first level of customer support with this AI chat (and before you get upset, know that the first level of customer support is currently a very bad rule-based AI). Main website in Finnish: https://aatos.app https://aatos.app (also some services for SE and DK, plus we recently opened UK with just a e-sign service)

baobabKoodaa 3y ago
Here's an example of what our product recommendations look like: Given your ownership in a company and real estate, a lasting power of attorney is a prudent step. This allows you to appoint PARTNER_NAME or another trusted individual to manage your business and property affairs in the event of incapacitation. Additionally, it can also provide tax benefits by allowing tax-free gifts to your children, helping to avoid unnecessary inheritance taxes and secure the financial future of your large family.
Closi 3y ago
> Since we already have the users' legal documents available to us as a result of our core business, we are perfectly positioned to build supplementary AI chat features related to legal documents. Uhh... What are the privacy implications here?!
- baobabKoodaa 3y ago
  If you look at the example I posted of our product recommendations, you will see that the GPT-4 generated text contains "PARTNER_NAME" instead of actual partner name. That's because we've created anonymized dataset from users in such a way that it's literally impossible for OpenAI to deanonymize users. Of course the same can not be done if we want to provide a service where users can, for example, chat with their legal documents. In that case we will have to send some private details to OpenAI. Not sure how that will pan out (what details we decide to send and what we decide not to send). In any case, all startups today are created on top of a mountain of cloud services. Any one of those services can leak private user data as a result of outsider hack or insider attack or accident. OpenAI is just one more cloud service on top of the mountain.

nostromo 3y ago

And not just legal either. I uploaded all of my bloodwork tests and my 23andme data to Chat GPT and it was better at analyzing it than my doctor was.

slingnow 3y ago
This is a really interesting use case for me. I've been envisioning a specially trained LLM that can give useful advice or insights that your average PCP might gloss over or not have the time to investigate. Did you do anything special to achieve this? What were the results like?
p-e-w 3y ago
Yup. Because the doctor doesn't have time and doesn't give a fuck. LLMs don't have to compete against the cutting edge of human professional knowledge. They only have to compete against the disinterested, arrogant, greedy, and overworked professionals that are actually available to people in practice. No wonder they're winning.