4 ms·

Your last statement hints at a big consideration: accountability. One lawyer on a formerly 15 lawyer staff is accountable for 15 lawyers worth of potential mist

Your last statement hints at a big consideration: accountability. One lawyer on a formerly 15 lawyer staff is accountable for 15 lawyers worth of potential mistakes, and we know that “but the AI did it!” doesn’t hold water in law.

DannyBee 2y ago
There's a bunch of assumptions here. The main one i think is probably wrong is that there was 15 lawyers worth of work being done before (when measured by some average lawyer standard). For example, it's possible there was only really 1 lawyer worth of work being split 15 ways, so each lawyer was really only responsible for 1/15th of an average lawyers amount of work :) In that scenario, they'd only be responsible for 1 average lawyers worth of mistakes now. Is that realistic? Who knows. I've definitely seen that level of "waste" (for lack of a better term) before in law firms :) Even in the scenario you are positing, it's not obvious it matters as much as you seem to think it does. If the per-lawyer mistake rate was low enough, it may be that 15x that rate simply does not matter. These kinds of contracts are fairly standardized, and so they are mostly looking at the differences from last time. Those differences are often not legal as much as factual. IE the table of costs changed, not the legal responsibility. So the main thing mistakes get you is maybe cost (if mistakes matter at all). This isn't like they are seeing brand new from scratch contracts constantly that require brand new analysis. Even if they were, like I said, the main issue with a mistake is cost. For all we know, the AI company also agreed to indemnify them for a certain rate of mistakes or something (which wouldn't be hard to get insurance for). I'm not actually a fan of AI taking necessary jobs, but I think the view here that this is sort of life or death is strange. I'd be much more worried about AI handling criminal defense in some semi-autonomous fashion than this.
- chefandy 2y ago
  > There's a bunch of assumptions here. Undoubtedly. Happy to be disabused of my misgivings. > The main one i think is probably wrong is that there was 15 lawyers worth of work being done before (when measured by some average lawyer standard). For example, it's possible there was only really 1 lawyer worth of work being split 15 ways, so each lawyer was really only responsible for 1/15th of an average lawyers amount of work :) > In that scenario, they'd only be responsible for 1 average lawyers worth of mistakes now. Is that realistic? Who knows. I've definitely seen that level of "waste" (for lack of a better term) before in law firms :) Even in the scenario you are positing, it's not obvious it matters as much as you seem to think it does. > If the per-lawyer mistake rate was low enough, it may be that 15x that rate simply does not matter. Well having done quite a bit of work with attorneys no longer practicing law, I’m definitely familiar with the gripes about inefficiencies and running up hours— especially during litigation in the larger firms. Even not being as efficient as they could be, assuming 1400% inefficiency or whatever seems much less reasonable than assuming 0% inefficiency. It’s obviously not either of those extremes, but I have a hard time imagining it’s even close to the former. > These kinds of contracts are fairly standardized, and so they are mostly looking at the differences from last time. Those differences are often not legal as much as factual. IE the table of costs changed, not the legal responsibility. So the main thing mistakes get you is maybe cost (if mistakes matter at all). > This isn't like they are seeing brand new from scratch contracts constantly that require brand new analysis. Even if they were, like I said, the main issue with a mistake is cost. > I don’t actually know what kind of contracts they were working on so I’ll have to take your word on that. > For all we know, the AI company also agreed to indemnify them for a certain rate of mistakes or something (which wouldn't be hard to get insurance for). I was involved with the AI legal tool scene indirectly for about a decade, but haven’t been for a couple years, and am only getting info indirectly from people I know that still are. (Actually clicking through the top results on Google, I’m actually on a first-name basis with the first founder there was a picture of. I didn’t know he started a new company though so I guess we’re not THAT close!) My knowledge could be out of date, but I’ve not seen one of these services offer indemnity for mistakes and ostensibly for good reason — the latest data I’ve seen shows that attorney-targeted legal tools make more mistakes than people hoped. I also know nothing about legal insurance, but I don’t think it would be smart to insure an organization that just canned 94% of their counsel in favor of tools known to not be particularly reliable when their workload probably has not changed. Whether they did it because either they care more about payroll than reliability, or they had the poor judgement to maintain 1500% staffing levels until then, it still seems like a pretty poor bet. > I'm not actually a fan of AI taking necessary jobs, but I think the view here that this is sort of life or death is strange. I certainly don’t think it’s life or death, and of all the places in our society that could use a little more efficiency, legal services is right up there. That said, the fact that it’s not life or death also doesn’t mean that it’s totally fine either. > I'd be much more worried about AI handling criminal defense in some semi-autonomous fashion than this. Haha— frankly, I don’t give a damn if the hospital signs a terrible contract that costs them a bazillion dollars as long as they don’t pull a Steward and stop purchasing basic medical supplies. SURELY public defenders are an attractive target for the outright person-replacing sort of efficiencies, but I have a hard time imagining that would pass muster. I can definitely see some supposedly adversarial plea agreement system being implemented by more authoritarian jurisdictions as an incremental expansion of the NN sentence-recommendation type of tools. My gut says the bigger semi-automation risk there is overworked public defenders’ being lulled into false confidence in legal and general office LLM type tools (messages summaries, auto scheduling appointments, etc) without having the time to give them the scrutiny they need. I’d be shocked if that wasn’t already happening though. Hey maybe with a bunch of attorneys having newfound time on their hands they can bone up on criminal law and provide some relief for the public defender staffing crisis.

DannyBee 2y ago

There's a bunch of assumptions here. The main one i think is probably wrong is that there was 15 lawyers worth of work being done before (when measured by some average lawyer standard). For example, it's possible there was only really 1 lawyer worth of work being split 15 ways, so each lawyer was really only responsible for 1/15th of an average lawyers amount of work :) In that scenario, they'd only be responsible for 1 average lawyers worth of mistakes now. Is that realistic? Who knows. I've definitely seen that level of "waste" (for lack of a better term) before in law firms :) Even in the scenario you are positing, it's not obvious it matters as much as you seem to think it does. If the per-lawyer mistake rate was low enough, it may be that 15x that rate simply does not matter. These kinds of contracts are fairly standardized, and so they are mostly looking at the differences from last time. Those differences are often not legal as much as factual. IE the table of costs changed, not the legal responsibility. So the main thing mistakes get you is maybe cost (if mistakes matter at all). This isn't like they are seeing brand new from scratch contracts constantly that require brand new analysis. Even if they were, like I said, the main issue with a mistake is cost. For all we know, the AI company also agreed to indemnify them for a certain rate of mistakes or something (which wouldn't be hard to get insurance for). I'm not actually a fan of AI taking necessary jobs, but I think the view here that this is sort of life or death is strange. I'd be much more worried about AI handling criminal defense in some semi-autonomous fashion than this.

chefandy 2y ago
> There's a bunch of assumptions here. Undoubtedly. Happy to be disabused of my misgivings. > The main one i think is probably wrong is that there was 15 lawyers worth of work being done before (when measured by some average lawyer standard). For example, it's possible there was only really 1 lawyer worth of work being split 15 ways, so each lawyer was really only responsible for 1/15th of an average lawyers amount of work :) > In that scenario, they'd only be responsible for 1 average lawyers worth of mistakes now. Is that realistic? Who knows. I've definitely seen that level of "waste" (for lack of a better term) before in law firms :) Even in the scenario you are positing, it's not obvious it matters as much as you seem to think it does. > If the per-lawyer mistake rate was low enough, it may be that 15x that rate simply does not matter. Well having done quite a bit of work with attorneys no longer practicing law, I’m definitely familiar with the gripes about inefficiencies and running up hours— especially during litigation in the larger firms. Even not being as efficient as they could be, assuming 1400% inefficiency or whatever seems much less reasonable than assuming 0% inefficiency. It’s obviously not either of those extremes, but I have a hard time imagining it’s even close to the former. > These kinds of contracts are fairly standardized, and so they are mostly looking at the differences from last time. Those differences are often not legal as much as factual. IE the table of costs changed, not the legal responsibility. So the main thing mistakes get you is maybe cost (if mistakes matter at all). > This isn't like they are seeing brand new from scratch contracts constantly that require brand new analysis. Even if they were, like I said, the main issue with a mistake is cost. > I don’t actually know what kind of contracts they were working on so I’ll have to take your word on that. > For all we know, the AI company also agreed to indemnify them for a certain rate of mistakes or something (which wouldn't be hard to get insurance for). I was involved with the AI legal tool scene indirectly for about a decade, but haven’t been for a couple years, and am only getting info indirectly from people I know that still are. (Actually clicking through the top results on Google, I’m actually on a first-name basis with the first founder there was a picture of. I didn’t know he started a new company though so I guess we’re not THAT close!) My knowledge could be out of date, but I’ve not seen one of these services offer indemnity for mistakes and ostensibly for good reason — the latest data I’ve seen shows that attorney-targeted legal tools make more mistakes than people hoped. I also know nothing about legal insurance, but I don’t think it would be smart to insure an organization that just canned 94% of their counsel in favor of tools known to not be particularly reliable when their workload probably has not changed. Whether they did it because either they care more about payroll than reliability, or they had the poor judgement to maintain 1500% staffing levels until then, it still seems like a pretty poor bet. > I'm not actually a fan of AI taking necessary jobs, but I think the view here that this is sort of life or death is strange. I certainly don’t think it’s life or death, and of all the places in our society that could use a little more efficiency, legal services is right up there. That said, the fact that it’s not life or death also doesn’t mean that it’s totally fine either. > I'd be much more worried about AI handling criminal defense in some semi-autonomous fashion than this. Haha— frankly, I don’t give a damn if the hospital signs a terrible contract that costs them a bazillion dollars as long as they don’t pull a Steward and stop purchasing basic medical supplies. SURELY public defenders are an attractive target for the outright person-replacing sort of efficiencies, but I have a hard time imagining that would pass muster. I can definitely see some supposedly adversarial plea agreement system being implemented by more authoritarian jurisdictions as an incremental expansion of the NN sentence-recommendation type of tools. My gut says the bigger semi-automation risk there is overworked public defenders’ being lulled into false confidence in legal and general office LLM type tools (messages summaries, auto scheduling appointments, etc) without having the time to give them the scrutiny they need. I’d be shocked if that wasn’t already happening though. Hey maybe with a bunch of attorneys having newfound time on their hands they can bone up on criminal law and provide some relief for the public defender staffing crisis.