13 ms·

Top lawyers beaten by legal AI

237 points by eaguyhn 8y ago

pcunite 8y ago
Computers can compute numbers faster than 20 professors too ... can we please stop the AI is everything?
- njharman 8y ago
  Write AI that better curates news items.
pmlnr 8y ago
There's a plethora of science fiction that using only AI in court _will_ go wrong.
- AllegedAlec 8y ago
  With an emphasis on fiction.
- atrus 8y ago
  There's also a plethora of writings showing that using only using humans in court _will and has_ go wrong. There's probably a plethora of writings showing that using both humans and AI _will_ go wrong. The question isn't _will_ they go wrong (they will) it's a question of will they do it less.
- kurtisc 8y ago
  Nothing in the linked article is about using AI in court.
- deleted 8y ago
  [deleted]
deleted 8y ago
[deleted]
nickelcitymario 8y ago
Having worked in a law office, I can say confidently that the future of legal AI is human+AI. AI can rapidly spot all the issues, saving expensive lawyers from spending their time wading through the documents. But the human lawyer brings strategy, judgement and face-to-face consultation to the process. In other words, I don't think lawyers are in danger. But law clerks? Junior staff? The people who are usually forced to do all the grunt work to allow the top lawyers to focus on their strengths? Those jobs are at definitely on the chopping block. There's still an insane amount of rote repetitive work in your standard law firm.
- atonse 8y ago
  What I've heard (but many years ago) is that the legal profession in general is extremely tech-hostile (some is just cultural, but some is that it gets in the way of billable hours). I don't know how much of that has changed now, especially with younger lawyers growing up with tech, getting in and wanting to do things a certain way.
  - fredley 8y ago
    This is one of the things where the market will decide. All it takes is one firm to drastically reduce costs while maintaining standards (or possibly improving them?) and the rest will be forced follow suit.
  - rayiner 8y ago
    That has not been my experience. The industry is highly competitive. Each matter might be pitched to half a dozen firms. If tech does shave costs, that can be a huge differentiating factor. For the most part though it doesn’t. There are many clients who won’t pay for first year associates. For those clients, firms don’t roll out legal tech. They do the work and write it off. Even at top wall street firms realization is 80-90% hours billed. Nor do firms trot out legal tech when it comes to contigency or fixed-fee cases, where there are no billable hours. Some major clients these days demand arrangements where they pay a fixed. monthly price. There is no legal tech leveraged in those cases. For the most part I’ve found software in general these days to be underwhelming. It took me years to find software that I trust enough to keep track of highlighted points in court opinions. (Shout out to the folks who make Citavi.) I remember writing about predictive coding in law school almost ten years ago. I have never used it on a case. Ironically, the only cases where there were enough documents (millions) for predictive coding to make sense also involved so much at stake the client wanted to spare no expense. The highest value legal tech has been review platforms like Relativity, but they are also crap. Like, loads documents slower than you can read them crap. It’s like technology to help programmers. You still just use Emacs because all this visual IDE shit doesn’t really help.
ax0ar 8y ago
The green header is very repulsive and distracting. I found it difficult to read the story.
- theonemind 8y ago
  Firefox's Reader mode works pretty well for things like that. uBlock Origin also has an eye-dropper to pretty easily block selected elements. You can typically remove floating headers like that. This bookmarklet also usually kills sticky headers: https://alisdair.mcdiarmid.org/kill-sticky-headers/ https://alisdair.mcdiarmid.org/kill-sticky-headers/ Anyway, common problem. You can probably find a stock solution.
overthemoon 8y ago
The labor cost savings aren't going to make it to clients. I've worked in a law office, I know how billing goes. Those extra billable hours will be found elsewhere. I hope I'm wrong, and maybe I'm just cynical, but my opinion of lawyers and the legal profession in general took a hit after 3 years of working in it.
- scotty79 8y ago
  You are right. Price of every item has nothing to do with the cost of making it and everything with how much people agree to pay for it. Driving down the cost of making a thing doesn't lower the price for everyone, but it widens the market downwards serivicing new el-cheapo customer base.
DanHulton 8y ago
I just joined a company last year that works in this space almost exactly - due diligence contract review - and we are hearing this pretty constantly from people who evaluate our software. Not only is it much faster, but frequently it finds clauses that they had missed themselves. It's a pretty exciting time to be working in this space.
paulgrant999 8y ago
"Predictive Coding". You don't need AI. You just need collusion between law firms i.e. to write a program, and then agree, that whatever it interprets, is the way it will be semantically enforced in court. To be clear: the study of law is bullshit; so why not take advantage of it, and simply agree, that computer generated bullshit, is what determines it... ... of course while still retaining high rates and billable hours for "attorney review" (for billable hours/padding).
- paulgrant999 8y ago
  and once you get the Courts ok'ing it, then why do you even need lawyers or courts? If no-one is going to look at the actual evidence/agreements/law....
- deleted 8y ago
  [deleted]
mv4 8y ago
I've always felt that a good contract should look like code - that it should "compile" and "run" without errors.
- and0 8y ago
  Agreed! And I'm thinking that some sort of language (or document format with technical markup) is going to become normalized to take advantage of this software, shifting from pure "prose".
- Skrillex 8y ago
  I can imagine system with drag and drop sentence fragments and modifiable variables designed for the construction of contracts. When a document is finished, it would produce a summary of the consequences with as few interpretations as possible, possibly with a toggle for Wikipedia-style keyword linking in the result. The document could then be tweaked until the summary exactly matches what both parties want, at which point the document and summary together become the contract. Although, if such a system were to become widely used, I feel like lawyers will still be used to either discover or introduce any other remotely plausible interpretations that exist in the generated contract.
  - deleted 8y ago
    [deleted]
caffeine5150 8y ago
I am a transactional lawyer and I definitely would find value in an application that could issue spot an agreement in seconds. That said, just yesterday I spoke on a panel on the topic of how things can go wrong in a contract. We spent the majority of time talking about the dynamics and challenges that exist outside the agreement in the process of trying to memorialize the parties’ intent in a clear, concise, precise and reasonably complete manner. There are often significant challenges in terms of clearly obtaining the intent and relevant issues from the various stakeholders. And there are dynamics like relative negotiating leverage and psychology or other issues that can drive what the deal will look like regardless of pure legal issues. Also, since one never starts with a blank page, there is the contract template one starts with that must be evaluated against all this – what stays, what goes, what must change and how. Navigating these requires intangible skills, instincts, sensitivity to human dynamics, etc. It’s very much a human endeavor. So a key question is to what extent AI could help with all of these external issues. I have to think that’s much farther down the road. But having help assessing purely legal issues within the document would be a great supplement. For a good thread on training to be a lawyer through apprenticeship, see this https://news.ycombinator.com/item?id=16255023 https://news.ycombinator.com/item?id=16255023
caffeine5150 8y ago
51-156 minutes is crazy to review an NDA. If an NDA is well drafted, I can do it in about 10 minutes. If it's a bit of a mess, maybe 30 min. If it's worse than that, I can assess that in about 5 minutes and propose using a better form.
- deleted 8y ago
  [deleted]
- cf141q5325 8y ago
  Its the number for all 5 NDAs combined, so fitting exactly with your numbers.
  - caffeine5150 8y ago
    Ah, thank you for the correction.
dctoedt 8y ago
With an AI, you don't know what it is that the AI doesn't know. A trivial example: Some "nondisclosure" agreements (NDAs) also include invention-assignment agreements [0] and/or non-competition covenants. I'll recognize such provisions if I see one in a putative NDA, but I don't know whether the AI will recognize it. Sure, the AI could provide a list of everything that it does know, but when I review the list, I won't necessarily notice that a missing item is missing. [0] Stanford University got trapped by an invention-assignment agreement contained in an "NDA" that one of its researchers signed when he visited someone at Roche — see https://en.wikipedia.org/wiki/Stanford_University_v._Roche_Molecular_Systems,_Inc https://en.wikipedia.org/wiki/Stanford_University_v._Roche_M....
- deleted 8y ago
  [deleted]
- JumpCrisscross 8y ago
  > I don't know whether the AI will recognize it This sounds like the easier work. Capture what you expect and flag every deviation for human review.
- caffeine5150 8y ago
  This is an important point, although I can imagine AI could be trained to spot things that don't belong or are unusual provisions in an NDA. This may be what you're getting at, but it's easy in a contract review to focus on reacting to what's there and it's harder to know what's missing. An NDA is a relatively simple, cookie-cutter types of agreement with widely agreed elements. Other agreements not so much. How would AI figure out what's missing?
  - dctoedt 8y ago
    > How would AI figure out what's missing? That's an important point, but I assume an AI would use some sort of checklist, perhaps seeded by human lawyers, to spot expected-but-missing provisions.
sandworm101 8y ago
>> It took the lawyers an average of 92 minutes to complete the NDA issue spotting, compared to 26 seconds for the LawGeex AI. Issue-spotting an NDA is not a typical task for an actual lawyer. It is the sort of thing handled by a legal intern or paralegal... then signed off on by a lawyer. It isn't typical legal work. The results are therefor of limited application in the real world. Better test: Client asks "Do I need an NDA?" or, conversely, "can I break this NDA I signed last year?". Ask C3PO to handle that question. That is the sort of problem where the real legal expertise happens.
- kemitchell 8y ago
  > Ask C3PO to handle that question. I had a good laugh here. Thanks!
- acct1771 8y ago
  That decision comes down to the specifics of the case - which could be turned into parameters.
joshuaheard 8y ago
Clicking on the link, I expected to discover a watershed moment akin to AI beating a human at Chess or Go. Alas, since contract review is not an adversarial process, no human lawyers were "beaten". A less click-baity headline would be: AI slightly more efficient at issue-spotting contracts than lawyers.
- Digory 8y ago
  And "issue spotting" here means "identify if there are unusual clauses in this NDA." When lawyers talk about "issue spotting," they usually mean "given these facts, what legal principles apply?" This is not that.
  - kemitchell 8y ago
    > When lawyers talk about "issue spotting," they usually mean "given these facts, what legal principles apply?" This is not that. Accurate.
RichardBurtLaw 8y ago
NDAs tend to be a straightforward type of document with limited scope and a fairly well-known universe of issues. So having a computer program parse the document and itemize issues seems like a good idea. But what does it mean that the AI “beat” the humans? That the program simply spotted the issues and listed them faster and more completely than the lawyers? Not that impressive a victory. It’s how the issues get resolved that are important. Did the AI interact with a lawyer on the other side and get issues resolved? Not likely, but the story doesn’t tell us exactly what the test consisted of and how “victory” was determined. (It is possible to download the 40-page report and analyze it, but I am not willing to spend the time to do that, and I bet most commenters, like me, are just responding to the headline take-away.) While AI can be a great tool, one of the problems with it is that often no one knows how the program makes decisions. Not knowing what you don’t know can present serious blind spots. For example, it may be possible for a counter-party to a transaction who has access to the same AI as you to construct a document with text containing an issue that the AI won’t spot but that the counter-party planted in the document for his or her advantage. If the document is reviewed only by AI (and given a cursory review by a bored low-paid worker who expects AI to catch all the issues), then there is an open back door to your legal castle. Computers haven’t put accountants out of business, and they won’t put lawyers out of business either. They will make lawyers more efficient, but whether that translates into lower fees remains to be seen.
- kemitchell 8y ago
  Greetings from up north, in Oakland. Your points on "beating" and decision making are very well taken. Same on the resilience of the professions. My own take here is basically that it's accommodating inefficient behavior---a thousand implementations of the same basic deal terms---rather than correcting it. The cure for the common NDA is standardization, which is something the profession can do for itself. Along those lines, I recently launched an initiative to stop pointless waste on review of routine MNDAs: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ The idea was not to write the world's greatest NDA, or to offer whiz-bang features with software, but to create a boring, unsurprising, neutral NDA with a distinctive name that spreads word about itself, creating opportunities to sign routine NDAs without waiting for redundant legal review. Would love to read your thoughts, here or by e-mail.
casper345 8y ago
"Top mathematicians were beaten by calculator AI that computes equations faster"
jacobkg 8y ago
Is this technology somewhat akin to a code linter? It automatically finds that problems that would be tedious for humans to find and greatly speeds up the work of the human expert?
tshadley 8y ago
Statement from Professor Yonatan Aumann, advisor to Tel Aviv-based LawGeex, on LawGeex AI: “The technology has been developed through a combination of supervised and unsupervised learning techniques. Unsupervised learning was used for teaching the AI engine the core legalese language. Thereafter, supervised learning, using deep learning multi-layer LSTM and convolution technology, was used to train the system for the fine-tuned issue-spotting. Supervision was performed based on human-annotated documents, using legal experts. A unique augmentation algorithm was applied to boost learning from these examples. The overall result is the most advanced technology for the automatic analysis of legal documents. The p-value for the statement that accuracy of AI is above that of these lawyers is 0.0068 (using MannWhitney’s U test). https://images.law.com/contrib/content/uploads/documents/397/5408/lawgeex.pdf https://images.law.com/contrib/content/uploads/documents/397...
buboard 8y ago
Lawyers have traditionally been able to secure their own payout by law, so they ll just thakn you for the extra free time , but will this lead to actual reductions of legal costs?
- kemitchell 8y ago
  I'm a solo transactional attorney, and I launched a "viral" NDA designed to get routine deals signed without legal review a couple weeks ago: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ I don't want another dime for reviewing standard mutual NDAs. There is way, way better work for me to do.
browsercoin 8y ago
this is interesting point in time for me....im thinking about becoming a lawyer....im caught between my love for machine learning and deep learning and the desire to practice law.
YeGoblynQueenne 8y ago
Can I be a skeptic? The study was carried out by the people who sell the software. How reliable is that?
kemitchell 8y ago
Time wasted reviewing pointless variations on the same old mutual NDA is a scourge. It's hard to take the profession's performative technophilia seriously while the most obvious, widespread inefficiency continues every day. On the other hand, teaching computers to parse pointless variations of the same substantive terms in legalese accommodates bad practice, instead of correcting it. It's symptomatic treatment at best, enablement at worst. Lawyers shouldn't be sending and receiving a thousand different NDAs for the same fundamental deal. They should be defining and evolving a shared vocabulary of common forms, and invoking them whenever cost-benefit favors a standard. That's the idea behind a very recent project of mine, The Canting Tribe NDA: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ It's a "viral" NDA. The website publishes a versioned form. The form includes a signed certificate guaranteeing that the proposal is identical to the version published on the site. On the site, newcomers see a list of firms using and accepting the form. The experience shows, rather than tells, how standardization can work. I soft launched the project just a couple weeks ago. A couple firms signed on almost immediately. Several more are reviewing. My top priority is more firms on the website. Once the site provides enough baseline social proof, the form can spread word and create new users on its own.
- fblp 8y ago
  This is great. Would be interested to integrate with an e-signing service. Configuring standard forms for e-signing is also a scourge. Any competitors to this? NDA reads clean...
  - kemitchell 8y ago
    Funny you should mention! This is my third attempt at "The NDA Problem". Attempt 1: Very modern, plain-language NDA. Much easier to read and follow. Substantively better. No uptake. Attempt 2: Suite of NDAs for various use cases, integrated with e-signature, filing, attorney approval, the works. That was rxnda.com. Lots of positive feedback. Not much actual use. Attempt 3: Canting Tribe. Zero code. All about the network effects. Much, much simpler. I've tried to make the PDF copy of the form as easy to set up in place-the-blanks e-sig services as possible, with [bracketed instructions] to make clear what goes where. I would be very interested in an e-sig service that allowed me to preconfigure a template with the latest terms, for other people to use. I'd be very interested in ways to put metadata or other hints for setting up e-sig into any of the various formats the site publishes. I used to have a list of companies offering something close to rxnda.com: NDA plus e-sig. I don't have it with me now. But I'm not aware of any other attempt at voluntary standardization of a general-purpose NDA. Several large companies with leverage have published their standard NDAs, or even set up e-sig portals, as Google has. But I'm not aware of any such forms offered for general use, between other companies. I would not be surprised to learn of prior attempts, or approaches vaguely close to mine. Frankly, I don't care who solves this problem, or how, as long as it's solved. Canting Tribe is the best I have. I believe it can work, but I can't make it work alone.
known 8y ago
Soft skills?

237 points by eaguyhn 8y ago

pcunite 8y ago

Computers can compute numbers faster than 20 professors too ... can we please stop the AI is everything?

njharman 8y ago
Write AI that better curates news items.

pmlnr 8y ago

There's a plethora of science fiction that using only AI in court _will_ go wrong.

AllegedAlec 8y ago
With an emphasis on fiction.
atrus 8y ago
There's also a plethora of writings showing that using only using humans in court _will and has_ go wrong. There's probably a plethora of writings showing that using both humans and AI _will_ go wrong. The question isn't _will_ they go wrong (they will) it's a question of will they do it less.
kurtisc 8y ago
Nothing in the linked article is about using AI in court.
deleted 8y ago
[deleted]

deleted 8y ago

[deleted]

nickelcitymario 8y ago

Having worked in a law office, I can say confidently that the future of legal AI is human+AI. AI can rapidly spot all the issues, saving expensive lawyers from spending their time wading through the documents. But the human lawyer brings strategy, judgement and face-to-face consultation to the process. In other words, I don't think lawyers are in danger. But law clerks? Junior staff? The people who are usually forced to do all the grunt work to allow the top lawyers to focus on their strengths? Those jobs are at definitely on the chopping block. There's still an insane amount of rote repetitive work in your standard law firm.

atonse 8y ago
What I've heard (but many years ago) is that the legal profession in general is extremely tech-hostile (some is just cultural, but some is that it gets in the way of billable hours). I don't know how much of that has changed now, especially with younger lawyers growing up with tech, getting in and wanting to do things a certain way.
- fredley 8y ago
  This is one of the things where the market will decide. All it takes is one firm to drastically reduce costs while maintaining standards (or possibly improving them?) and the rest will be forced follow suit.
- rayiner 8y ago
  That has not been my experience. The industry is highly competitive. Each matter might be pitched to half a dozen firms. If tech does shave costs, that can be a huge differentiating factor. For the most part though it doesn’t. There are many clients who won’t pay for first year associates. For those clients, firms don’t roll out legal tech. They do the work and write it off. Even at top wall street firms realization is 80-90% hours billed. Nor do firms trot out legal tech when it comes to contigency or fixed-fee cases, where there are no billable hours. Some major clients these days demand arrangements where they pay a fixed. monthly price. There is no legal tech leveraged in those cases. For the most part I’ve found software in general these days to be underwhelming. It took me years to find software that I trust enough to keep track of highlighted points in court opinions. (Shout out to the folks who make Citavi.) I remember writing about predictive coding in law school almost ten years ago. I have never used it on a case. Ironically, the only cases where there were enough documents (millions) for predictive coding to make sense also involved so much at stake the client wanted to spare no expense. The highest value legal tech has been review platforms like Relativity, but they are also crap. Like, loads documents slower than you can read them crap. It’s like technology to help programmers. You still just use Emacs because all this visual IDE shit doesn’t really help.

ax0ar 8y ago

The green header is very repulsive and distracting. I found it difficult to read the story.

theonemind 8y ago
Firefox's Reader mode works pretty well for things like that. uBlock Origin also has an eye-dropper to pretty easily block selected elements. You can typically remove floating headers like that. This bookmarklet also usually kills sticky headers: https://alisdair.mcdiarmid.org/kill-sticky-headers/ https://alisdair.mcdiarmid.org/kill-sticky-headers/ Anyway, common problem. You can probably find a stock solution.

overthemoon 8y ago

The labor cost savings aren't going to make it to clients. I've worked in a law office, I know how billing goes. Those extra billable hours will be found elsewhere. I hope I'm wrong, and maybe I'm just cynical, but my opinion of lawyers and the legal profession in general took a hit after 3 years of working in it.

scotty79 8y ago
You are right. Price of every item has nothing to do with the cost of making it and everything with how much people agree to pay for it. Driving down the cost of making a thing doesn't lower the price for everyone, but it widens the market downwards serivicing new el-cheapo customer base.

DanHulton 8y ago

I just joined a company last year that works in this space almost exactly - due diligence contract review - and we are hearing this pretty constantly from people who evaluate our software. Not only is it much faster, but frequently it finds clauses that they had missed themselves. It's a pretty exciting time to be working in this space.

paulgrant999 8y ago

"Predictive Coding". You don't need AI. You just need collusion between law firms i.e. to write a program, and then agree, that whatever it interprets, is the way it will be semantically enforced in court. To be clear: the study of law is bullshit; so why not take advantage of it, and simply agree, that computer generated bullshit, is what determines it... ... of course while still retaining high rates and billable hours for "attorney review" (for billable hours/padding).

paulgrant999 8y ago
and once you get the Courts ok'ing it, then why do you even need lawyers or courts? If no-one is going to look at the actual evidence/agreements/law....
deleted 8y ago
[deleted]

mv4 8y ago

I've always felt that a good contract should look like code - that it should "compile" and "run" without errors.

and0 8y ago
Agreed! And I'm thinking that some sort of language (or document format with technical markup) is going to become normalized to take advantage of this software, shifting from pure "prose".
Skrillex 8y ago
I can imagine system with drag and drop sentence fragments and modifiable variables designed for the construction of contracts. When a document is finished, it would produce a summary of the consequences with as few interpretations as possible, possibly with a toggle for Wikipedia-style keyword linking in the result. The document could then be tweaked until the summary exactly matches what both parties want, at which point the document and summary together become the contract. Although, if such a system were to become widely used, I feel like lawyers will still be used to either discover or introduce any other remotely plausible interpretations that exist in the generated contract.
- deleted 8y ago
  [deleted]

caffeine5150 8y ago

I am a transactional lawyer and I definitely would find value in an application that could issue spot an agreement in seconds. That said, just yesterday I spoke on a panel on the topic of how things can go wrong in a contract. We spent the majority of time talking about the dynamics and challenges that exist outside the agreement in the process of trying to memorialize the parties’ intent in a clear, concise, precise and reasonably complete manner. There are often significant challenges in terms of clearly obtaining the intent and relevant issues from the various stakeholders. And there are dynamics like relative negotiating leverage and psychology or other issues that can drive what the deal will look like regardless of pure legal issues. Also, since one never starts with a blank page, there is the contract template one starts with that must be evaluated against all this – what stays, what goes, what must change and how. Navigating these requires intangible skills, instincts, sensitivity to human dynamics, etc. It’s very much a human endeavor. So a key question is to what extent AI could help with all of these external issues. I have to think that’s much farther down the road. But having help assessing purely legal issues within the document would be a great supplement. For a good thread on training to be a lawyer through apprenticeship, see this https://news.ycombinator.com/item?id=16255023 https://news.ycombinator.com/item?id=16255023

caffeine5150 8y ago

51-156 minutes is crazy to review an NDA. If an NDA is well drafted, I can do it in about 10 minutes. If it's a bit of a mess, maybe 30 min. If it's worse than that, I can assess that in about 5 minutes and propose using a better form.

deleted 8y ago
[deleted]
cf141q5325 8y ago
Its the number for all 5 NDAs combined, so fitting exactly with your numbers.
- caffeine5150 8y ago
  Ah, thank you for the correction.

dctoedt 8y ago

With an AI, you don't know what it is that the AI doesn't know. A trivial example: Some "nondisclosure" agreements (NDAs) also include invention-assignment agreements [0] and/or non-competition covenants. I'll recognize such provisions if I see one in a putative NDA, but I don't know whether the AI will recognize it. Sure, the AI could provide a list of everything that it does know, but when I review the list, I won't necessarily notice that a missing item is missing. [0] Stanford University got trapped by an invention-assignment agreement contained in an "NDA" that one of its researchers signed when he visited someone at Roche — see https://en.wikipedia.org/wiki/Stanford_University_v._Roche_Molecular_Systems,_Inc https://en.wikipedia.org/wiki/Stanford_University_v._Roche_M....

deleted 8y ago
[deleted]
JumpCrisscross 8y ago
> I don't know whether the AI will recognize it This sounds like the easier work. Capture what you expect and flag every deviation for human review.
caffeine5150 8y ago
This is an important point, although I can imagine AI could be trained to spot things that don't belong or are unusual provisions in an NDA. This may be what you're getting at, but it's easy in a contract review to focus on reacting to what's there and it's harder to know what's missing. An NDA is a relatively simple, cookie-cutter types of agreement with widely agreed elements. Other agreements not so much. How would AI figure out what's missing?
- dctoedt 8y ago
  > How would AI figure out what's missing? That's an important point, but I assume an AI would use some sort of checklist, perhaps seeded by human lawyers, to spot expected-but-missing provisions.

sandworm101 8y ago

>> It took the lawyers an average of 92 minutes to complete the NDA issue spotting, compared to 26 seconds for the LawGeex AI. Issue-spotting an NDA is not a typical task for an actual lawyer. It is the sort of thing handled by a legal intern or paralegal... then signed off on by a lawyer. It isn't typical legal work. The results are therefor of limited application in the real world. Better test: Client asks "Do I need an NDA?" or, conversely, "can I break this NDA I signed last year?". Ask C3PO to handle that question. That is the sort of problem where the real legal expertise happens.

kemitchell 8y ago
> Ask C3PO to handle that question. I had a good laugh here. Thanks!
acct1771 8y ago
That decision comes down to the specifics of the case - which could be turned into parameters.

joshuaheard 8y ago

Clicking on the link, I expected to discover a watershed moment akin to AI beating a human at Chess or Go. Alas, since contract review is not an adversarial process, no human lawyers were "beaten". A less click-baity headline would be: AI slightly more efficient at issue-spotting contracts than lawyers.

Digory 8y ago
And "issue spotting" here means "identify if there are unusual clauses in this NDA." When lawyers talk about "issue spotting," they usually mean "given these facts, what legal principles apply?" This is not that.
- kemitchell 8y ago
  > When lawyers talk about "issue spotting," they usually mean "given these facts, what legal principles apply?" This is not that. Accurate.

RichardBurtLaw 8y ago

NDAs tend to be a straightforward type of document with limited scope and a fairly well-known universe of issues. So having a computer program parse the document and itemize issues seems like a good idea. But what does it mean that the AI “beat” the humans? That the program simply spotted the issues and listed them faster and more completely than the lawyers? Not that impressive a victory. It’s how the issues get resolved that are important. Did the AI interact with a lawyer on the other side and get issues resolved? Not likely, but the story doesn’t tell us exactly what the test consisted of and how “victory” was determined. (It is possible to download the 40-page report and analyze it, but I am not willing to spend the time to do that, and I bet most commenters, like me, are just responding to the headline take-away.) While AI can be a great tool, one of the problems with it is that often no one knows how the program makes decisions. Not knowing what you don’t know can present serious blind spots. For example, it may be possible for a counter-party to a transaction who has access to the same AI as you to construct a document with text containing an issue that the AI won’t spot but that the counter-party planted in the document for his or her advantage. If the document is reviewed only by AI (and given a cursory review by a bored low-paid worker who expects AI to catch all the issues), then there is an open back door to your legal castle. Computers haven’t put accountants out of business, and they won’t put lawyers out of business either. They will make lawyers more efficient, but whether that translates into lower fees remains to be seen.

kemitchell 8y ago
Greetings from up north, in Oakland. Your points on "beating" and decision making are very well taken. Same on the resilience of the professions. My own take here is basically that it's accommodating inefficient behavior---a thousand implementations of the same basic deal terms---rather than correcting it. The cure for the common NDA is standardization, which is something the profession can do for itself. Along those lines, I recently launched an initiative to stop pointless waste on review of routine MNDAs: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ The idea was not to write the world's greatest NDA, or to offer whiz-bang features with software, but to create a boring, unsurprising, neutral NDA with a distinctive name that spreads word about itself, creating opportunities to sign routine NDAs without waiting for redundant legal review. Would love to read your thoughts, here or by e-mail.

casper345 8y ago

"Top mathematicians were beaten by calculator AI that computes equations faster"

jacobkg 8y ago

Is this technology somewhat akin to a code linter? It automatically finds that problems that would be tedious for humans to find and greatly speeds up the work of the human expert?

tshadley 8y ago

Statement from Professor Yonatan Aumann, advisor to Tel Aviv-based LawGeex, on LawGeex AI: “The technology has been developed through a combination of supervised and unsupervised learning techniques. Unsupervised learning was used for teaching the AI engine the core legalese language. Thereafter, supervised learning, using deep learning multi-layer LSTM and convolution technology, was used to train the system for the fine-tuned issue-spotting. Supervision was performed based on human-annotated documents, using legal experts. A unique augmentation algorithm was applied to boost learning from these examples. The overall result is the most advanced technology for the automatic analysis of legal documents. The p-value for the statement that accuracy of AI is above that of these lawyers is 0.0068 (using MannWhitney’s U test). https://images.law.com/contrib/content/uploads/documents/397/5408/lawgeex.pdf https://images.law.com/contrib/content/uploads/documents/397...

buboard 8y ago

Lawyers have traditionally been able to secure their own payout by law, so they ll just thakn you for the extra free time , but will this lead to actual reductions of legal costs?

kemitchell 8y ago
I'm a solo transactional attorney, and I launched a "viral" NDA designed to get routine deals signed without legal review a couple weeks ago: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ I don't want another dime for reviewing standard mutual NDAs. There is way, way better work for me to do.

browsercoin 8y ago

this is interesting point in time for me....im thinking about becoming a lawyer....im caught between my love for machine learning and deep learning and the desire to practice law.

YeGoblynQueenne 8y ago

Can I be a skeptic? The study was carried out by the people who sell the software. How reliable is that?

kemitchell 8y ago

Time wasted reviewing pointless variations on the same old mutual NDA is a scourge. It's hard to take the profession's performative technophilia seriously while the most obvious, widespread inefficiency continues every day. On the other hand, teaching computers to parse pointless variations of the same substantive terms in legalese accommodates bad practice, instead of correcting it. It's symptomatic treatment at best, enablement at worst. Lawyers shouldn't be sending and receiving a thousand different NDAs for the same fundamental deal. They should be defining and evolving a shared vocabulary of common forms, and invoking them whenever cost-benefit favors a standard. That's the idea behind a very recent project of mine, The Canting Tribe NDA: https://nda.cantingtribe.com/ https://nda.cantingtribe.com/ It's a "viral" NDA. The website publishes a versioned form. The form includes a signed certificate guaranteeing that the proposal is identical to the version published on the site. On the site, newcomers see a list of firms using and accepting the form. The experience shows, rather than tells, how standardization can work. I soft launched the project just a couple weeks ago. A couple firms signed on almost immediately. Several more are reviewing. My top priority is more firms on the website. Once the site provides enough baseline social proof, the form can spread word and create new users on its own.

fblp 8y ago
This is great. Would be interested to integrate with an e-signing service. Configuring standard forms for e-signing is also a scourge. Any competitors to this? NDA reads clean...
- kemitchell 8y ago
  Funny you should mention! This is my third attempt at "The NDA Problem". Attempt 1: Very modern, plain-language NDA. Much easier to read and follow. Substantively better. No uptake. Attempt 2: Suite of NDAs for various use cases, integrated with e-signature, filing, attorney approval, the works. That was rxnda.com. Lots of positive feedback. Not much actual use. Attempt 3: Canting Tribe. Zero code. All about the network effects. Much, much simpler. I've tried to make the PDF copy of the form as easy to set up in place-the-blanks e-sig services as possible, with [bracketed instructions] to make clear what goes where. I would be very interested in an e-sig service that allowed me to preconfigure a template with the latest terms, for other people to use. I'd be very interested in ways to put metadata or other hints for setting up e-sig into any of the various formats the site publishes. I used to have a list of companies offering something close to rxnda.com: NDA plus e-sig. I don't have it with me now. But I'm not aware of any other attempt at voluntary standardization of a general-purpose NDA. Several large companies with leverage have published their standard NDAs, or even set up e-sig portals, as Google has. But I'm not aware of any such forms offered for general use, between other companies. I would not be surprised to learn of prior attempts, or approaches vaguely close to mine. Frankly, I don't care who solves this problem, or how, as long as it's solved. Canting Tribe is the best I have. I believe it can work, but I can't make it work alone.

known 8y ago

Soft skills?